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I. 



INTRODUCTION 



A . GENERAL 

Assume that all the resources needed for a particular 
production process are already purchased. They represent sunk 
costs to the production facility. The only cost of doing 
business then is the cost incurred by producing unacceptable 
goods. If finished goods are flawed or late when delivered 
to the consumer in a market economy then the producer 
eventually loses market share. Continued loss of market share 
leads to annihilation of the production business. Thus, the 
decision maker's task is to allocate the available resources 
to achieve timely delivery of error-free products. This is 
the situation in which some mid-level managers find 
themselves. In other words, given little or no control over 
the resources on hand, they are to perform as well as possible 
but are not to fail. 

The typical response to the mid-level manager's production 
problem is to employ part of the allotted labor resources as 
quality assurance inspectors. Use of labor resources as 
inspectors rather than producers decreases production. The 
time required to conduct the inspections slows the remaining 
production. The use of the inspectors, however, decreases the 
number of flaws in the finished product. The manager faces 
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a tradeoff between two favorable objectives: better finished 

products versus more finished products. 

Assume that all worker errors in the production process 
that can lead to a flawed finished product can be identified. 
Assume that each worker error^ (where i = l,...,m) is 
independent and mutually exclusive of each other. Therefore, 
given that a flawed product is found bearing the effects of 
one error from the production process: 

(1-1) Z: P (error.) = 1.0 

i = l 

Assume a cost distribution can be identified for error. 
From those distributions assume that mean costs can be 
derived. Now the expected cost of doing business (EC) per 
each item produced can be written: 

(1-2) Z P (error.) * C (error.) = EC 

i = l 

where m is the number of independent errors 
P is "probability" 

C is "mean cost" 

The probability for a flawed finished product due to error 
on taskj^ [ P ( error^ ) ] is the probability that both the 

worker erred on the task and the inspector failed to identify 
the worker’s error. The inspector’s ability to positively 
identify flawed products as being unacceptable increases over 
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time. Naturally, the cost of doing business can be reduced 
if the allocation of available inspector time can be driven 
toward errors that occur more frequently and which have 
greater cost. How should the available inspection time be 
divided between error-prone tasks to best reduce the cost of 
doing business? The focus of this thesis is how to allocate 
the available inspection time to best accomplish the manager's 
multiple objectives of speed and thoroughness. 

Goal programming is a convenient tool for simultaneously 
satisfying several objectives. The first half of this thesis 
proposes a methodology for allocating inspection time based 
on goal programming. The second half uses the artillery gun 
chief's inspection problem to illustrate the methodology. 

In offering a solution to the allocation of inspection 
time problem, the thesis uses several timely, efficient, and 
sound techniques to derive a deterministic probability for 
worker failure on each error-prone task, the expected time 
that an inspector spends inspecting each task, and the cost 
distribution of error on each task. The hope is that these 
techniques are sufficiently convenient that they can be 
readily adopted by mid-level managers in their allocation of 
inspection resource problems. 
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B. DEFINITIONS 



Some terminology is peculiar to the study of human 
reliability. Other terminology is peculiar to the U.S. Army 
and the field artillery. Appendix F contains a glossary of 
those terms. 

C. ORGANIZATION OF THE THESIS 

Chapter 2 presents the experimental results and theories 
that are the underpinnings of the proposed methodology for 
solving the allocation of inspection time problem. The 
chapter documents alternative techniques by which error rates 
and time estimates may be derived other than by empirical 
observation. The chapter reviews a technique for estimating 
costs of errors and discusses the attributes of goal 
programming . 

Chapters 3 through 5 present the problem that illustrates 
the proposed methodology. Chapter 3, Background to the 
Artillery Problem, describes the allocation of inspection time 
problem as it applies to the gun section and the artillery 
fire support system. The discussion of artillery explains the 
tasks of the various crewmen and the errors they make. 
Chapter 4 develops a model for the artillery gun chief's 
problem. Chapter 5 applies the methodology to an active army 
unit at Fort Ord, California. Chapter 6, Conclusions and 
Recommendations, proposes other applications of this 
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methodology and suggests areas for future research. The 
appendices give the output from the computer runs discussed 
in Chapter 5, the surveys of respondents, and step-by-step 
procedures for computing the solution to the artillery gun 
chief's inspection problem. 
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II. DEVELOPMENT OF A VISUAL INSPECTION MODEL 



A. GENERAL 

1 . Need for a Model over Intuition 

Often the allocation of inspection time to various 

inspection tasks is determined by the inspector. It is 

generally expected that an inspector will employ his full 

experience, knowledge, and intuition when deciding how to 

spend his time on the various visual inspection tasks he must 

perform. Intuition is inadequate for such tasks as allocating 

inspection time. Hogarth identifies the primary weakness in 

reliance on intuition in general: 

One could suppose that through intuitive judgment people 
are able to capture important information which escapes 
codification and is thus not amenable to statistical 
analysis. Indeed, intuitive judgment does capture much 
information, however, it is inconsistent. Moreover such 
inconsistency necessarily attenuates human predictive 
validity. In other words, although humans may on occasion 
exhibit high accuracy in judgment, inability to apply 
judgmental rules consistently across a series of cases 
means that on average models outpredict people. [Ref 2, 
p.194] 

Over the long run, a model that incorporates all available 
knowledge regarding inspection phenomenon and allocates 
inspection time accordingly will outperform any inspector's 
intuition. 
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A mathematical model of the inspection process offers 

many advantages over reliance on the intuition of an 

inspector. First, a model offers a means to understand the 

underlying causes and effects occurring in the inspection 

phenomena. Reliance on inspector's intuition offers no 

alternative method to solve inadequacies should intuition 

start failing. With a model there is the means to test 

alternative solutions. Gass states. 

It (a mathematical model) enables us to evaluate the 
effects of a change of one variable on all the other 
variables; it provides us with a quantitative basis to 
sharpen and evaluate our intuition of the process under 
investigation. . . Mathematical models enable us to bring 
some semblance of scientific methodology to areas of 
decision making heretofore characterized by intuition and 
experience. [Ref 42, p. 26-27] 

Morse and Kimball write, "Operations research is a 
scientific method of providing executive departments with a 
quantitative basis for decisions regarding operations under 
their control [Ref 19, p 7]." Building effective mathematical 
models of the inspection process allows managers to take 
control of their inspection resources and make optimal use of 
them in support of production facility goals and values. If 
inspectors are not brought in to support the goals and values 
of the production facility they are apt to pursue their own 
set of goals and values. Hillier and Lieberman allude to what 
then tends to happen "...(there is) a tendency for many 
components of an organization to grow into relatively 
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autonomous empires with their own goals and value systems [Ref 
5, p 3]." 

Inspector time is a resource of the manager and, as 
such, it should support the production facility's goals and 
values. If the manager can quantify his goals with respect 
to product quality then goal programming allows him to model 
his inspection time problem. The model gives him the means 
to simultaneously consider his goals and allocate inspector's 
time to satisfy those goals. 

2. Modeling of the Inspection Process 

The purpose of the inspection process is to identify 
worker errors that affect the system. The model of the 
inspection process must incorporate the effect that the errors 
potentially have on the system. Thus, a large part of 
modeling the inspection process is error analysis. 

The importance of an error is directly related to its 
potential for unacceptable system effects. The Nuclear 
Regulatory Commission (NRC) makes this point in regard to 
errors in general: 

The importance of an error is a function of its frequency, 
the efficiency of the (inspector) for the error, and the 
likelihood and severity of the potential consequences of 
the error... [Ref 14, p 4-17]. 

In applying this same concept to the realm of human error, 
Pickrel and McDonald write: 
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The extent and cost of efforts to eliminate sources of 
human errors should be commensurate with: 

1) the frequency with which the error is expected to 
occur; 

2) the frequency with which a (system) failure will 
occur as a result of the error; and 

3) the possible consequences of the failure 
condition. [Ref 27, p 649] 

Swain points the direction for the modeling of the inspection 
process when he writes: 

Errors are important only if they lead to important 
consequences. Thus, the steps in error analysis are 

a) Identify the consequences of interest; 

b) Identify all errors that can lead to those 
consequences ; 

c) Determine error rates for each class of errors; 

d) Identify the likely contributing factors. 

[Ref 10, p.l4] 

The probability of worker error, the probability of associated 
inspector error, and the cost of associated product flaws are 
the parameters needed to establish the relative importance of 
the different errors. These parameters must then be brought 
together in one model to represent the inspection process. 

This chapter reviews the rationale for using the 
opinions of subject-matter experts to both estimate the 
probability for worker error and to estimate the time required 
to inspect for that worker error. Theories and experiments 
dealing with the probability for inspector failure as a 
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dealing with the probability for inspector failure as a 
function of time to inspect and a method for cost estimation 
of system error are reviewed. 

Another area discussed in the model development is the 
importance of internal consistency in judgments made by 
subject-matter experts before their subjective evaluations 
have merit. Saaty's Analytical Hierarchy Process [Ref 17] is 
reviewed for its method of evaluating the internal consistency 
in a respondent. Also, in this chapter, goal programming is 
reviewed as a means to model problems such as the allocation 
of inspection resources problem. The goal program that 
achieves this allocation of inspector time is relatively 
simple and straightforward. The challenge is developing the 
probabilities for error and the costs of error required by the 
model and the goal program. 

B. THE PROBABILITY OF CREW ERROR 

1. Rationale for Subjective Evaluations 

Research into human reliability and error analysis led 
to work by the Nuclear Regulatory Commission (NRC) in its 
evaluation of human error probability in the operation of 
nuclear power plants. The NRC is at the forefront in research 
on human error and design of functional, safe systems that 
incorporate human error rates. 
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Traditional thinking is that personal opinion of 
people with working experience is worth little as a basis for 
good operations research work. Morse and Kimball were 
emphatic about this when they helped found operations 
research: 

The opinions of a few dozen persons who have had 
operational experience provide an extremely shaky 
foundation for any operations research. . .The need for 
unbiased, impersonal facts, not opinions, must always be 
borne in mind. [Ref 19, p 8-9] 

This influence is still clear today as there is often an 
aversion to using operational experience as a basis for 
operations research. Echoing the sentiments of Morse and 
Kimball is the following more recent comment by Von 
Winterfeldt and Edwards that typifies the views of many 
systems analysts about the appropriate way to assess 
probabilities : 

By far the most common practical procedure for assessing 
probabilities is to collect a large set of observations 
and then to use a relative frequency. This procedure is 
tedious but the existence of empirical science testifies 
to its success. [Ref 29, p. 110] 

Whereas probabilities for equipment failure may be 
best established by the strict empirical approach, it is 
unlikely that probabilities for human failure (human error) 
can be gathered in that manner. Swain and Miller provide a 
discussion of the reasons human error data generally cannot 
be obtained through empirical observations [Ref 1, p 226]. 
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Horley and Giordano give a graphic example of failure of the 
empirical approach to obtain human error data on artillery gun 
sections in U.S. Army tests conducted at Fort Hood, Texas [Ref 
15, p. 67-73]. Swain and Guttman discuss why the empirical 
approach to generating human error data has not worked in 
nuclear power plants: 

The collection of relative frequencies (for human error 
rates by empirical observation) is impractical for several 
reasons . 

First, error probabilities for many tasks are very 
small. Therefore, in any reasonable time period, not 
enough errors will occur to permit the fitting of a 
meaningful distribution to the observed error relative 
frequencies . 

Second, usually there is some penalty associated with 
the commission of an error, which discourages the 
identification of individuals who commit the errors. If 
such identification were insisted upon, there would be 
some unknown, but sizeable, number of errors not reported. 

Third, the administrative problems and costs of 
recording and analyzing errors by person would probably 
be unacceptable. [Ref 14, p. 6-2] 

There are numerous activities in need of human 
reliability data for enhanced man-machine system design. In 
most of these activities there is extensive operational 
experience, but there is little or no human error data. 
Expert judgment provides a means to generate human error data. 
Swain states. 

The use of collective judgment by subject-matter experts 
is one way to make up for the paucity of empirical error 
rate data. Studies indicate that it would be feasible to 
have a number of people with detailed knowledge of various 
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tasks in an industrial setting rank-order tasks in terms 
of error-likeliness . [Ref 10, p.lOl] 

Miller and Swain briefly describe how the methods that use 

expert judgment to generate human error data are structured: 

Subject-matter experts judge the relative or absolute 
likelihood of error for several task descriptions, and 
these responses are converted to usable human error 
probabilities by mathematical scaling techniques. [Ref 1, 
p. 238] 

The NRC endorses four methods for eliciting expert 

judgment input to estimate human error probabilities. The NRC 

makes the following exhortation in regards to these methods: 

Make the best use possible of current procedures and 
information, even though they may be f lawed. .. These (the 
four methods endorsed by the NRC) are based on sound 
psychological theory and empirical support that indicate 
their potential usefulness. And as a practical matter, 
they can be used now to obtain needed human error 
probabilities. [Ref 14, p 8-6] 

The NRC approved methods are the paired comparisons procedure, 
ranking and rating procedure, direct numerical estimation 
procedure, and indirect numerical estimation procedure. Swain 
and Guttman [Ref 14, p.8-6 to 8-13] and Seaver and Stillwell 
[Ref 28, p A-2 to A-51] provide detailed explanations of each 
of the NRC approved methods. 

2 . Experts to Supply Expert Judgment 

Embrey noted that his whole approach to developing 
human error probabilities by subjective input was contingent 
on judges. He writes: 

The success of the approach is likely to be critically 
dependent on the knowledge and experience that the judges 
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possess regarding the tasks they are asked to assess. For 
this reason it is very important that strenuous attempts 
are made to obtain appropriately qualified judges. [Ref 
28, p. 30] 

In commenting about eliciting judgments of uncertainty (or 
probabilities) from people. Von Winterfeldt and Edwards write: 



If a numerical measure of uncertainty is an opinion, it 
characterizes a person as well as an event; we want to 
elicit from the right person. Who is that?.. [Ref 29, p. 
113] 

The "right person" is the one who has the best understanding 
of the phenomenon under investigation and who can best relate 
that understanding in terms that can be quantified. 

Swain offers that the worker or inspector himself is 
an expert on error rates: 

Subject-matter experts are persons who are especially 
knowledgeable about the tasks for which error rates are 
to be derived. Thus, if one wanted to derive error rate 
estimates for new tasks in an assembly plant, the experts 
would be persons who perform or had performed similar 
tasks in an assembly plant. [Ref 10, p.lOl] 

Saaty agrees. Saaty not only allows that the worker or 

inspector is an expert in the operational aspects of his work, 

but he implies that he also is the best person for relating 

to others phenomena about that work: 

We generally agree that people who experience a phenomenon 
firsthand are the ones who can best shed light on our 
understanding of it; indeed, knowledge derived from 
experience is basic to all understanding. [Ref 17, p. 9] 
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Lai, Shenoi, and Fan suggest that workers and inspectors are 

specifically well-equipped to estimate probabilities 

concerning errors and hazards; 

One of the major problems facing many quantitative risk 
assessment studies is the lack of proper data about the 
probabilities of occurrence of hazardous events. Often, 
the only alternative is to employ rough estimates provided 
by operations personnel.... Human beings often possess 
quality knowledge that arises out of their insights and 
experiences. When the safety of a system is being 
examined this expertise has the greatest weight. [Ref 40, 
p.136] 

While the worker and inspector are certainly experts 
on the operational aspect of the human error phenomenon that 
is a subject of this study, the question remains whether they 
can relate this expert understanding in terms that permit 
quantification. 

3. Indirect Numerical Estimation Technique 

Miller and Swain write that a major disadvantage of 
methods that use expert judgment to generate human error data 
is that the expert often lacks sufficient familiarity with 
probabilities to be able to give meaningful input [Ref 1, p. 
244] . Persons with operational experience, such as workers 
and inspectors, are often not qualified to give a direct 
numerical estimate of a probability. Paulos states, "An 
appreciation for probability takes a long time to develop. 
[Ref 7, p.l33]." Morse and Kimball warn, "People seldom 

estimate random events correctly .. .their opinions are nearly 



15 



always unconsciously biased [Ref 19, p. 8]." Swain cautions: 

It is not suggested that (subject-matter) experts be asked 
to make estimates of actual error rates for tasks. Such 
estimates would tend to vary widely and would generally 
underestimate the true error rates. Some subject-matter 
experts simply would refuse to make such estimates. [Ref 

10, p 101] 

Hogarth states. 

People do have difficulty in expressing their degree of 
knowledge in the precise quantitative form demanded by 
probability theory.... Numerous studies have shown 

fallibilities in intuitive attempts to guess probabilistic 
relations from given facts. [Ref 2, p.l92] 

Embrey shows that, when giving uncertainty judgments, 

respondents do significantly better when dealing with 

likelihood ratios and odds rather than probabilities. Embrey 

concludes, "that judges (subject-matter experts) are best at 

making simple directional judgments of the form 'task A is 

more likely to be successful than task B' [Ref 28, p 9]." 

Swain's work supports Embrey 's. Swain adds: 

Persons who will not guess at the probability of a 
particular event will readily rank-order several events 
in terms of increasing difficulty, hazard, or other 
dimension, and they will do this ranking with great 
confidence. [Ref 10, p.lOl] 

Killer and Swain comment, "Humans are very capable of making 
qualitative judgments along one dimension [Ref 1, p. 238]." 

Estimation methods for human error probability that 
allow a person without probability-estimating skills to relate 
his expert understanding of error phenomena by simple 
directional judgments could be very useful. Of the four 
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expert judgment methods proposed by the NRC, two methods are 
based upon simple directional judgments: paired comparisons 

and indirect numerical estimation. 

A problem with paired comparisons is the large number 
of judges required to obtain valid estimates of the task 
positions on a subjective scale. The relative frequency by 
which one task is rated more likely to fail than another by 
a large pool of experts is what leads to a scaled value for 
that task. If there are only five judges providing input then 
there are insufficient observations to be significant. 

The indirect numerical estimation method of obtaining 
human error probabilities from expert judgement is a robust 
procedure and is the primary procedure for estimating human 
error rates pursued in this thesis. Like paired comparisons, 
it depends mostly on human qualitative versus quantitative 
judgments. The indirect numerical estimation method asks the 
respondent two questions concerning task failure in his area 
of expertise: 

1) Is this task error more or less likely than that task 
error? 

2) How much more or how much less likely is the first 
task error versus the second task error? 

A detailed discussion of the conversion of these expert 

judgments to human error data by the indirect numerical 
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estimation method is given by Seaver and Stillwell [Ref 12] . 
A summary of that process is given in the example below. 

Using the one-by-one comparisons given by a 
respondent, a subjective scale of likelihood is generated- 
Assume a respondent gives these likelihoods for four mutually 
exclusive, independent task errors that encompass the entire 
set of errors that can occur on a given system: 

Task A error is twice as likely as Task B error. 

Task B error is just as likely as Task C error. 

Task C error is three times as likely as Task D error. 

The likelihood of task error for all tasks can simply be 
written in terms of any one task error. In the example, use 
D as the base scale: 

Task A error = 6 X (Task D error) 

Task B error = 3 X (Task D error) 

Task C error = 3 X (Task D error) 

Task D error = 1 X (Task D error) 

If the probability of any one task error is known or estimated 
from some other source (Task D for example) then the 
probabilities of all four events are known. 

If the error probabilities are normalized, then the 
probability for error in any one task is known, given that an 
error occurs. Normalization takes place in the example 
problem : 
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6D + 3D + 3D + ID = 1.00 
D = .0769 

Therefore, given an error occurs on the system, the 
probability that it was: 

Task A error is 46% 

Task B error is 23% 

Task C error is 23% 

Task D error is 8% 

These probabilities for task error apply to one respondent. 

The next topic concerns the number of respondents 

needed to generate good point estimates of the error data 

being studied. Von Winterfeldt and Edwards state, "Both 

theory and evidence teach us to use more than one respondent 

whenever we can. [Ref 29, p.ll3)." Guttman and Swain, 

commenting on the indirect numerical estimation technique, 

cite the work of Von Holstein in stating, "as with direct 

numerical estimation, little is gained by using more than six 

experts [Ref 14, p A-46] . " Hogarth writes: 

...judgments between individuals are likely to be 
correlated. Thus one is often averaging redundant 
information with the result that beyond a certain point 
gains in adding experts to a group are small. It has been 
estimated that in most predictive situations involving the 
use of experts there is little point in averaging more 
than about ten persons and that six or seven will often 
be sufficient. [Ref 2, p 195] 

The next subject is the aggregating of several 
individual likelihood estimates to a single estimate 
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applicable to the entire production facility. Many schemes 

exist for combining several likelihood estimates into a single 

estimate. Von Winterfeldt and Edwards offers this insight: 

Since proper scoring rules are convex functions on the 
probability simplex, the score of the average of 
individual probabilities will necessarily be better than 
the average of the individual's scores. [Ref 29, p.l33] 

The authors continue with this straightforward recommendation: 

Simply elicit the desired uncertainty measures from each 
member individually. . . . Transform the results into 
probabilities and average them. The odds seem excellent 
that, if you do anything more complex, you will simply be 
wasting your effort. [Ref 29, p.l36] 

This thesis pursues the method recommended by Von Winterfeldt 

and Edwards . 



4. Consistency in Expert Judgment 

Workers and inspectors are typically trained to simply 



perform the tasks 


required in their jobs. 


They are not 


routinely 


trained 


to make 


quantitative 


judgments 


about 


phenomena 


relating 


to their 


work . When 


asked to 


make 



quantitative judgments it can be expected that some personnel 
are overwhelmed with the requirement and thus fail to provide 
valid input. It would be helpful to have a means to screen 
the incoming data to eliminate surveys of those respondents 
who were incapable or unwilling to provide valid data. 

Consistency on the part of a respondent is some 
measure of objectivity, internal logic, and operational 
experience. Saaty defines the term: 
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The meaning of consistency is that the intensities of 
relations among ideas or objects based on a particular 
criterion justify each other in some logical way. (Ref 
17, p 18) 

Some measure of consistency is necessary to establish the 

credibility of the expert whose judgments will eventually 

affect a final decision. Saaty states, "We do not want the 

decision to be based on judgments that have such low 

consistency that they appear random (Ref 17, p.82)." 

Stillwell and Seaver add, "High levels of inconsistency would 

suggest the entire judgmental process is suspect (Ref 13, 

p.32)." A way to identify inconsistent respondents is to 

require them to make more pairwise ratio judgments than 

absolutely necessary and then use those judgments to check the 

respondent's internal consistency. Seaver and Stillwell 

describe how this works for the indirect numerical estimation 

method of obtaining human error probabilities: 

To obtain a complete set of probabilities for N events 
only requires a minimum of N-1 such judgments and an 
independent estimate of the human error probability of 
one event. Additional judgments, however, may be obtained 
as consistency checks. [Ref 12, p A-46] 

When more pairwise ratio judgments are made by a respondent 

than is absolutely necessary there will most likely be some 

inconsistency. Stillwell and Seaver remind that, "Some 

inconsistencies enter into all judgmental processes. (Ref 13, 

p.32)" Saaty argues that some amount of inconsistency is 
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necessary as humans adapt to an ever-changing world. Saaty 
states : 

We may not be perfectly consistent, but that is the way 
we tend to work. It is also the way we grow. When we 
integrate new experiences into our consciousness, previous 
relationships may change and some consistency is lost. 
[Ref 17, p 82] 

A level of consistency is desirable to establish 
respondent validity and expertise. A level of inconsistency, 
however, is to be expected among human judgment. Saaty 
comments, "As long as there is enough consistency to maintain 
coherence among the objects of our experience, the consistency 
need not be perfect [Ref 17, p.82]." There is a definite 
need to establish an acceptable level of inconsistency in a 
judge, beyond which the judgments are discarded as invalid. 

To obtain some measure of consistency Saaty suggests 
the development of an (N X N) reciprocal matrix where N 
represents the number of items for comparison. A respondent 
makes a ratio comparison between every two possible items such 
that a total of (N) X (N-l)/2 judgments are made. The (N X 
N) matrix is filled in with ones down the main diagonal, the 
(N) X (N-l)/2 judgments fill the lower triangular matrix below 
the diagonal of ones, and the reciprocals of the judgments 
fill the upper triangular matrix above the diagonal of ones. 

The maximum eigenvalue (A„ax) this matrix is a 
measure of consistency of the judgments in the matrix. Saaty 
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proves that, "A positive reciprocal matrix is consistent if 
and only if is equal to N. (Ref 16, p.181)" Saaty then 
turns into a consistency index (Cl) by the following 
operation: 

(2-1) [ X„„-N ] / [ N-1 ] = Cl. 

Saaty suggests that comparing the Cl of a set of respondents' 
judgments against that Cl found by a randomly generated 
reciprocal matrix should give a general measure of the 
consistency called the consistency ratio (CR): 

( ^irespondent ) / ( ^irandoni matrix) ~ CR . 

The random entries in the random reciprocal matrix are chosen 
by computer from the range of possible values used by the 
respondents on their original survey. Several hundred random 
matrices are computed with respective CIs for each. Those CIs 
are then averaged to obtain a mean Cl for a random matrix. 
Appendix D shows the table for judgments limited to a one to 
nine scale and describes the generation of the table in more 
detail . 

Saaty provides a general rule for acceptable 
consistency ratio values with which he had success in his 
work. Saaty maintains that a consistency ratio less that 0.1 
demonstrates acceptable consistency and that a respondent 
providing such consistency is giving judgments worth 
considering in decision-making. [Ref 17, p. 83] 
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A disadvantage of Saaty's method is the number of 
judgments required of each respondent. In a 4 X 4 matrix an 
expert makes only six judgments [(4 X 3) / 2]. In a 10 X 10 
matrix an expert must make 45 judgments [(10 X 9) / 2]. Saaty 
suggests extensive use of hierarchies so that the number of 
judgments are reduced. For example, if the 10 X 10 matrix 
could be partitioned to sets of four, three, and three for 
pairwise ratio comparisons then the number of judgments is 
reduced from 45 to 15. Figure 1 depicts the reduction of 
required judgments that is accomplished by use of a hierarchy. 

Saaty's method is the method pursued in this thesis 
for establishing sufficiently consistent input for 
consideration in decision making. 

C. THE PROBABILITY OF INSPECTOR ERROR 
1. Inspectors Make Mistakes 

Traditionally inspectors are viewed as a panacea for 
production problems. Many production and quality assurance 
plans are premised on an assumption of 100% accuracy on the 
part of inspectors. Inspectors are not perfect. Swain makes 
this clear. 

There is no point in arguing whether people should make 
errors. We all do. Human error rates, like material 
failure rates, are merely facts [Ref 10, p.5]. 
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Total Comparison© Required - 45 




Total Comparisons 



Required 






N = (n) X (n-1)/2 



Where N • numbf'f of oornoeflnone needed 
to eelebliRti interval eoele 
using the Psirwiee Comparloons Method. 

n • number of Items to be 
pul In Interval sosle. 



Figure 1 - Reduction of Pairwise Judgments 
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Drury makes this call to action: 

Intrinsically, man is rarely perfect in either detection 
or diagnosis. • .This has led in recent years to studies 
which have demonstrated the effects inspector's errors can 
have. . .and the need to take account of realistic values 
of inspector's errors. [Ref 9, p.ll] 

To decrease the error rate some form of data collection is 

necessary to make a quantitative assessment of inspector 

error. Bennett emphasizes this point: 

So what must be done to achieve quality objectives given 
that inspection error is to be ever present? The answer 
is simple. The quality control engineer must be able to 
accurately measure these errors and then be able to design 
for them. [Ref 9, p.l] 

2. The Direct Relationship: the More Time Inspector 

Takes, the Greater the Inspector Accuracy 

An appealing method for viewing inspector error 

probability was introduced by Wreathall in work for the NRC. 

Wreathall’s Operator-Action Tree method (OAT) is depicted in 

Figure 2. 

The OAT approach is based on the assumption that human 
response to an environmental event consists of three 
activities : 

1) perception, 

2) diagnosis, and 

3) response. 

The basic operator-action tree is based on the potential 
for error in each of the three activities. 

The second major assumption is that time available for 
diagnosis is the dominant factor determining the 
probability for failure. That is, given a short period 
of time, people will fail to diagnose a situation 
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correctly more often than when given a longer period. [Ref 1, 
p.236] 



BASIC OPERATOR ACTION TREE 

OPERATOR OPERATOR OPERATOR 

EVENT OBSERVES DIAGNOSES CARRIES OUT SUCCESS/ 

OCCURS INDICATIONS PROBLEM REOUIRED RESPONSE FAILURE 

— 8 

F 

L F 

F 

SOURCE: NUREG/CR-3010 

Figure 2 - Operator-Action Tree 
The intuitive appeal for the first assumption is overwhelming. 
An inspector must "see", or visually fixate, on the event. 
He must be trained sufficiently to be able to discriminate 
error in the object of his fixation. Finally, the inspector 
must be able to respond properly to overcome the error. 
Failure to accomplish any one of these three steps will result 
in inspector failure on the task. 

Not only does the second assumption, that less time 
available means more inspector errors, make intuitive sense, 
but there is strong experimental basis for it. Drury's work 
with flat glass inspectors [Ref 26, p.265]. Buck's experiments 
with Landolt rings [Ref 9, p.l69], and Rizzi's tests with 

flawed dot patterns on cards [Ref 25, p.279] convincingly 

demonstrate that inspector errors increase with less exposure 
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time to the object being inspected. Drury summed up his work 

and that of Smith and Barany [Ref 22] when he wrote: 

In all of the studies, a similar effect is observed, in 
that as more time is allowed to inspect each item, the 
probability of rejecting a faulty item increases whilst 
the probability of accepting a good item decreases. [Ref 
32, p. 6] 

Drury suggests something akin to OAT: 

The inspection mechanism postulated is a visual search 
until either a potential defect is found or time runs out 
for that item, followed by a decision about the potential 
defect as to whether it is acceptable or rejectable. [Ref 
32, p. 6] 

Just as the OAT method suggests, a visual search must take 
place until the inspector "sees" the prospective error. This 
success of the visual search is no doubt a function of many 
factors. The experimental evidence reviewed points strongly 
to time available to the inspector as the most significant 
factor to whether the search will be successful or not. If 
the search leads to a prospective error then the inspector 
must decide whether to accept or reject the object bearing the 
prospective error. This decision involves experience of the 
inspector, remaining time available to the inspector, and 
costs and payoffs relating to finding errors. 

Smith and Barany found signal detection theory helpful 
in describing the decision an inspector must make [Ref 22, 
p.300]. Drury also used signal detection theory to model 
inspector decision performance [Ref 32, p.2-6]. Chapman and 
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Sinclair give this opinion of the application of signal 

detection theory to their study of inspection work: 

Perhaps the most important thing to emerge from this study 
is the usefulness of the theory of signal detection in 
industrial inspection situations both from a theoretical 
and a practical point of view. The theoretical value lies 
in the insights it provides into an inspection task, and 
the practical value arises from the fact that it allows 
economic justification for the application of ergonomics 
to inspection, and the rel-ative ease with which 
recommendations for improvement can be derived. [Ref 9, 
p. 241] 

The basics of signal detection theory were established 
by the pioneering work of Tanner, Swets, Birdsall, and Green 
between 1954 and 1966 [Ref 8, p.l8]. Figure 3 is the common 
depiction of the theory applied to the inspector's decision 
[Ref 9, p.l4]. The detection of signals is assumed by the 
theory to involve two processes: discrimination (d*) and 
decision (/?). Discrimination (d’) requires the inspector to 
be capable of distinguishing between success and error on a 
task he is inspecting. 

Discrimination is affected by such things as the 
experience of the inspector (more experience tends to make d' 
larger) and the time available to perform the inspection (more 
time tends to make d' larger). Decision (yS) requires the 
inspector to accept or reject what he sees. A decision (where 
the point 0 is between the two modal values of the Gaussian 
curves) is influenced by such things as the inspector's 
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Figure 3 - Signal Detection Theory [Ref 9, p.l4] 
subjective probability that an error will occur (if he thinks 
errors are very likely then he is more likely to reject) and 
his attitude toward risk. [Ref 8, p. 18-21] 

Smith and Barany explain in terms of signal detection 
theory what transpires to cause a reduction in inspector 
accuracy with a decrease in time available. Smith and Barany 
write : 

If the pace of the inspection task is increased, then the 
inspector will have less time in which to make his 
observation, and his observing mode will tend toward that 
described as "blurred observing" . [Ref 22, p.300] 

The effect of "blurred observing" is to decrease 

discriminability (d'). With discriminability (d') reduced 

there is more” overlap of the two Gaussian curves (see Figure 

3) and whatever decision (p) is made must result in either 
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more Type I or Type II errors. Smith and Bar any conducted 
experiments which demonstrated what signal detection theory 
postulated. 

The experimental results from Drury [Ref 26]; Rizzi, 
Buck, and Anderson [Ref 25]; and Smith and Barany [Ref 22] 
noted above are for relatively simple inspection tasks along 
one dimension (flaws in sheet glass, numbers of holes in 
disks, dots on cards) as opposed to more complex tasks with 
a multitude of possible errors on several dimensions (circuit 
boards, silicon chips). For simple inspection tasks the 
experimental results clearly show a direct relationship 
between inspector time to observe and inspector accuracy. 
Schoonard emphasized this point with an experiment where seven 
different classes of errors were possible on circuit boards. 
The inspection tasks for the seven classes were ranked from 
simplest to most difficult. A deliberate slowing of the 
inspection time resulted in marked improvement on the more 
simple inspection tasks but not so great an improvement on the 
other, more difficult inspection tasks. [Ref 24, p. 368-376] 
Drury predicted this finding by signal detection theory 
principles. In reference to simple inspection tasks Drury 
wrote , 

For simple discriminations (such as the Smith and Barany 
experiments) the discriminability will be high enough to 
give almost perfect inspection as the time allowed is 
increased. [Ref 32, p.7] 
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It is clear from experimental results that simple inspection 
task accuracy improves with time. 

The nature of the relationship between time taken by 
inspector and his accuracy was explored by both Rizzi, Buck, 
and Anderson [Ref 25, p. 279] and Drury [Ref 26, p.265]. 
Drury found an exponential relationship between the 
cumulative probability of locating a flaw in sheet glass and 
time taken to inspect. Rizzi, Buck, and Anderson conducted 
tests on subjects inspecting small cards with dots. Similar 
to Drury's results, Rizzi, Buck, and Anderson also found the 
inspector's exposure time to the object exerted an exponential 
effect on the probability of a correct detection. Figure 4 
shows the results of Drury's and Smith and Barany's work with 
the effect of time on the probability of inspector accuracy. 




Drury [Ref 26, p.264] Smith and Barany [Ref 32, p.5] 

Figure 4 - Inspector Accuracy as a Function of Time 
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3. Limits to the Direct Relationship 

It would seem that there must be some limit to the 
direct relationship that more time spent by an inspector 
yields more inspector accuracy. This section reviews 
experimental results that confirm both lower and upper bounds 
to this direct relationship. 

a. Minimum Time is Limited by Time Required for 
Visual Fixation 

The lower bound for this direct relationship is 
the physical limitation of the human eye to focus and fixate 
on any visual stimuli and the brain to comprehend what the eye 
is seeing. Experimental results revealing what might be a 
minimum time for the eye to fixate and the brain to begin to 
function for inspection purposes are offered by Drury. Drury 
concluded from his experimental results that it took seven 
tenths of a second (700 milliseconds) for his subjects to open 
and focus their eyes and begin searching for errors. [Ref 26, 
p. 260] . During that initial time from zero to seven tenths 
of a second there is no increase in inspector accuracy. 

b. Maximum Time is Limited by Human Vigilance 
The upper bound for the direct relationship of 

more time resulting in more accuracy is the result of 
decreased vigilance that occurs when an inspector exceeds his 
attention span. Chapman and Sinclair [Ref 9], Kochhar and 
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Jaisingh [Ref 23], and Schoonard, Gould, and Miller [Ref 24] 
demonstrated upper bounds to the direct relationship that was 
found by Drury [Ref 26]; Rizzi, Buck, and Anderson [Ref 25]; 
and Smith and Barany [Ref 22] . Experiments by Chapman and 
Sinclair of experienced personnel inspecting poultry carcasses 
at a meat processing plant [Ref 9, p.231] and the work of 
Kochhar and Jaisingh with experienced inspectors checking 
products on a CRT screen [Ref 23, p. 44] are two examples of 
note. Both showed initial improvement in inspector accuracy 
as time was increased, just as Drury and others had noted. 
At a point, however. Chapman and Sinclair [Ref 9] and Kochhar 
and Jaisingh [Ref 23] reported a degradation in inspector 
accuracy when inspectors were made to take more time in their 
visual inspection tasks. Schoonard, Gould, and Miller 
reported similar findings with their experienced inspectors 
who were checking circuit boards for any of seven different 
classes of errors that might be present. As Schoonard, Gould, 
and Miller increased the inspection time to 1.5 times the 
standard time the inspectors spent on their tasks, significant 
improvement resulted across all seven classes of error-types. 
With any additional increases over the 1.5 times the standard 
time for the inspectors, there was no improvement in inspector 
accuracy. [Ref 24, p.365]. Chapman and Sinclair offer a 
possible explanation for this degradation phenomenon with 
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increase in inspector time: 

. . .during the course of such a task periods of inattention 
occur which become more frequent or prolonged as 
presentation time increases. ... A possible answer (for the 
prolonged periods of inattention ) may be that the human 
organism maintains a homeostatic level of arousal, and if 
this is dependent on incoming stimuli, it could be argued 
that at the lower speeds additional stimuli are required 
which are obtained during the extended periods of 
inattention to the task. [Ref 9, p 249] 

Kochhar and Jaisingh offer this explanation of the decline in 

inspector accuracy resulting from increased inspection time 

available that was noted in their experiments: 

Error detection in vigilance situations of this type is 
dependent upon operator arousal levels. Thus the low 
levels of fault information and product pacing did not 
arouse the subject to the point where performance could 
be optimal.... It could be inferred that operator 
performance in inspection is related to arousal and in 
evaluating trade-offs between the different variables due 
consideration should be given to maintaining operator 
arousal. [Ref 23, p. 44] 

The phenomenon that more time beyond a certain critical point 
does not yield more inspector accuracy is real . Analysts 
cannot blindly assume that more inspector time will continue 
to yield an improving probability for accuracy. To continue 
to allot more inspector time is counterproductive. In 
addition to an actual decrease in inspector accuracy, more 
inspector time costs the facility either the salary of more 
inspectors or reduced productivity due to a slowed production 
line . 
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4. Significance of Schoonard ' s Findings 

Many visual inspection experiments have been 

conducted. Few of those experimental findings have direct 

application to the actual inspection operations on production 

lines. Schoonard, Gould, and Miller explain the shortcomings 

of the bulk of visual inspection experiments: 

Fundamental laboratory studies. . . have limited application 
to most actual inspection situations because they use 
discrete targets on homogenous backgrounds, whereas most 
inspection situations consist of poorly defined targets 
on non-homogenous backgrounds. [Ref 24, p 365] 

Another shortcoming of much experimental work is the 
hiring of students to perform the simple visual inspection 
tasks in the laboratory studies. The nature of most actual 
inspection tasks is sufficiently technical to require 
specially trained inspectors. The inspectors' frames of mind, 
motivation, and dedication have much to do with how they will 
perform when inspecting. The results will very likely differ 
significantly between experiments where fully paid and trained 
inspectors perform technical inspection tasks that they 
perform everyday and other experiments where students are paid 
some nominal fee to perform very simplistic visual inspection 
tasks for a day. To extrapolate information from experimental 
work for use in analysis of actual inspection work, the 
experiment should closely resemble the actual inspection 
situation in several key respects. 
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5 . Estimating Time for Inspector to Inspect 

An estimate on the standard time that it takes an 
inspector to perform a visual inspection task is necessary to 
establish the range of time over which an inspector can 
productively be employed on a task. In regards to eliciting 
estimates of times required to inspect for worker error the 
NRC writes: 

Estimation of performance times poses a more difficult 
problem (than human error probabilities). The best 
approach is to take measurements of actual or simulated 
task times. A second-best approach is to combine 
estimates of those who perform the tasks. [Ref 14, p. 6- 
1 ] 

Nothing precludes empirical measurement to derive the required 
time estimates of visual inspection tasks except the cost and 
difficulty of doing so. There may be several potential worker 
errors within the visual scope of the inspector at any one 
time. Measuring the time an inspector takes on any one visual 
task is complicated by the difficulty of the experimenter 
trying to track where the inspector's eyes are fixated. The 
costs of empirical measurement of times for visual inspection 
tasks is large due to the fact that each facility must likely 
derive its own time estimates due to individual facility 
peculiarities. Those peculiarities involve, for instance, 
level of training of the inspectors, special equipment that 
inspectors use, and standard operating procedures within the 
facility . 
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An alternative to empirical observation to derive time 
estimates on inspectors is a modification of the indirect 
numerical estimation technique. The standard inspector time 
on one task may be known through empirical observation. All 
other tasks are compared to one another on a respondent survey 
form. The respondents make directional judgments between 
pairs of tasks that appear on the survey form. The judgments 
call for: 

1 ) Which of the two visual inspection tasks requires more 
time to accomplish? 

2) About how much longer (in terms of seconds) does the 
one task take to inspect compared to the other task? 

Through simple algebraic manipulation estimates are derived 

from the survey as to the amount of time taken to perform all 

visual inspection tasks. This algebraic manipulation is given 

with sample data in Appendix C. These point estimates are 

averaged with the input of others from the same facility to 

yield a facility-wide estimate of time for an inspector to 

visually inspect every task. 

D. THE COSTS OF ERROR 

Many costs vie for the attention of the analyst and the 
executive. Swain writes; 

Most consideration must be given to those potential errors 
with an intolerable combined possibility of occurring, 
going undetected, and causing an unacceptable system 
consequence. [Ref 10, p.90] 
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An error that causes unacceptable system consequence is an 
error which is potentially too costly in terms of the 
consequences of interest for the system. Swain writes of the 
difference between errors that cause acceptable system 
consequence and those that cause unacceptable system 
consequence: 

For some errors, the system consequences may not be severe 
and perhaps a somewhat high possibility of an uncaught 
error can be tolerated. Other potential errors might have 
such severe consequences that uncaught error probabilities 
of even one in a million could not be tolerated. For 
example, a pharmaceutical company could probably not 
tolerate the existence of one cyanide capsule in one 
million multivitamin capsules delivered to the public. 
[Ref 10, p.89] 

The analyst must develop ways to assess potential costs of 
errors so that the manager can allocate error catching 
resources with regard to the potentially most damaging errors. 

The reallocation of error-catching resources in an 

organization is a change in system design. It is important 

to remember that in a systems approach: 

...successful design changes may not always reduce the 
probability of an error-but they will reduce the 
probability of undesirable system effects due to such 
errors. [Ref 10, p 90] 

In the multivitamin example mentioned by Swain, a very 
successful design change may be one that increases the 
incidence of cracked capsules (a frequent error of small 
costs) in bottles, but absolutely assures that a cyanide 
capsule (a rare error of monumental costs) never shows up in 
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a bottle of multivitamin capsules. It is essential that the 
potential costs of errors be estimated before resources can 
be allocated to reduce the probability of undesirable system 
effects . 

For any one type of error there is a corresponding 

distribution of costs. The U.S. Army suggests a quick method 

for establishing a cost distribution based on the Project 

Evaluating and Review Technique (PERT): 

One might well consider the possible use of subjective 
cost estimates based on PERT... 

Once it has been decided to employ the PERT technique 
to determine likely costs, it becomes necessary to 
estimate the most optimistic cost, the most pessimistic 
cost, and the most likely cost... 

These concepts fit in rather well with the idea of a 
generalized Beta distribution. An adequate statistical 
model of the Beta density to represent probable cost is: 



(2-3) f(c) = k(c - A)P * (B-c)*’, A<<<B 



where : 



f(c) = probability density function of costs 



c 



cost 



k 



constant to make area under distribution curve 
= 1 



(2-4) k = 1 / [(B-A)P“’*' * 0(p+l,q+l)] 



complete Beta function 



A 



minimum cost 



B 



maximum cost 



p,q = parameters determining the shape of the beta 
distribution [Ref 33, p.36-6] 
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From the PERT cost distribution a mean and standard deviation 



can be easily computed. 

(2-5) m = estimate of mean cost = (a + 4*mo + b) / 6 

(2-6) V = estimate of variance = (b - a)^ / 36 

where : 

a = estimate of minimum cost 

b = estimate of maximum cost 

m^ = estimate of most likely cost 
[Ref 33, p.36-7] 

See Appendix B for more details of this procedure. 

The PERT cost estimation technique requires expert opinion 

to generate estimates for the cost distribution. Moder and 

Phillips point to an immediate supervisor as one capable of 

providing estimates for PERT: 

The basis of PERT computations ... depends on the judgment 
of the person in charge of the activity in question. He 
is asked to call on his general experience, and his 
knowledge of the requirements of the activity in question, 
to consider the personnel and facilities available to him, 
and then to estimate the three (costs): optimistic, 

pessimistic, and most likely (costs). [Ref 34, p 204] 



E. GOAL PROGRAMMING 

1. Versatility and Utility 

Goal programming is a powerful mathematical 
programming technique which allows for models that very 
closely approximate problems in industry, business, and 
government. The power of goal programming is wrapped up in 
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its ability to overcome human analytical weaknesses. Paulos 

writes about one of those weaknesses: 

There's a strong human tendency to want everything, and 
to deny that trade-offs are usually necessary. . .Trade-offs 
between. . .speed and thoroughness. . .are frequently muddled 
and covered with a misty gauze, and this decline in 
clarity is usually an added cost for everyone. [Ref 7, 
p.l31] 

The decision maker can often visualize and announce what he 
wants to accomplish in several competing areas. The level at 
which the decision maker wants to achieve is a goal. The 
several goals that a decision maker has are often very 
different. Budnick, Mojena, and Vollman write of the power 
of goal programming: 

Goal programming ... allows for consideration of multiple 
goals. The goals may or may not be of the same dimension 
or unit of measurement. In addition, goal programming 
allows for consideration of conflicting goals. [Ref 6, p 
352] 



When it comes to giving something up for something 

else the decision maker often balks. Bell, Keeney, and Raiffa 

note the quandary in which the decision maker finds himself: 

This is what the multi-objective problem is all about: 
the making of vexing value trade-offs. There is no magic 
formula for making these value trade-offs. The decision 
maker would, of course, like to do the best he can in each 
attribute, but it is not possible to maximize several 
things at once. If we reduce unemployment, it may be at 
the expense of increasing inflation. . . These problems are 
pervasive and are at the heart of many public policy 
controversies. [Ref 35, p.4] 
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Any method that helps coax the decision maker into making 
tradeoffs is a help. Goal programming is a method for 
attacking the multi-objective problem. 

A decision variable is established which is the entity 
over which the decision maker has some control or influence. 
The decision variable is often a resource. The amount of the 
resource that can be employed is limited by constraints on the 
system. Through some known or hypothesized relationships 
varying amounts of the decision variable effect the levels of 
goal achievement. The relationships by which those decision 
variables affect the decision maker's goals are captured by 
mathematical models in the goal constraints. An achievement 
function mathematically combines the deviations from all 
goals. By minimizing or maximizing the achievement function 
through linear or nonlinear programming methods a level of 
resource (decision variable) use is determined such that the 
goals are most closely achieved. 

Goal programming calls for the satisfying of goals 
rather than the maximization of some single objective, as is 
often the case in standard linear programming. The decision 
maker states his goals, ranks them as to their importance, and 
weights them if necessary. The analyst determines the 
relationships between a controllable variable and the 
achievement of the goals. Goal programming gives the amount 
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of the controllable variable to expend to accomplish as 
closely as possible what the decision maker desires. 

Budnick, Mojena, and Vollman explain the satisfying 
of goals in this way: 

With multiple goals, all goals usually cannot be realized 
exactly. Goal programming attempts to minimize the 
deviations from these goals with consideration given to 
the hierarchy of stated priorities. [Ref 6, p 352] 

Ignizio refers to the greater applicability of multi-objective 

models to actual problems as a reason to pursue goal 

programming. Hillier and Lieberman present a concise review 

of goal programming [Ref 5, p 242-252] . 

2. Applied to Allocation of Inspection Time 

The overall allocation of inspection resources problem 
is one that is determined by executives in an organization. 
Take, for instance, an organization making an investment 
decision. A million dollar budget for quality assurance is 
being allocated. The decision of going with twenty visual 
inspectors or two visual inspectors and robotics is a decision 
far removed from the floor manager. It is not commonly within 
the manager's means to hire or fire inspectors or buy new 
inspection-aiding equipment. The manager must take what the 
organization provides him, whether it is two or twenty visual 
inspectors, and make the facility work to achieve quality and 
quantity standards. Basically, the only thing that the floor 
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manager may directly control in regard to inspection of the 
final product is the allocation of time that the inspector 
spends per visual task. 

a. Decision Variables 

The decision variables in goal programming are the 
variables over which the decision maker has some control and 
which have some influence over the achievement of the goals. 
In goal programming to achieve allocation of visual inspection 
resources the inspector time spent on checking the work of 
others is the decision variable ( X^, i= the number of visual 
inspection tasks). The purpose of goal programming is to 
determine an acceptable use of the decision variable. 

b. System Constraints 

A system is always limited in the amount of the 
decision variable that can be employed. This limitation is 
due to a limited supply of the resource or limited capacity 
of the system to employ the resource. In goal programming to 
allocate visual inspection time resources at least two system 
constraints exist: the minimum and the maximum amounts of 
time that can be spent productively on each of the inspection 
tasks. These limits are based on the experimental findings 
of Drury [Ref 26] and Schoonard, Gould, and Miller [Ref 24]. 
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c. Goal Constraints 



Goals reflect the objectives of the decision 
maker. With visual inspection there is usually a thoroughness 
goal and a speed goal that the manager specifies. These goals 
are elastic, allowing some deviation if insufficient resources 
are available to satisfy both goals. 

(2-7) Z Xi - TPOS + TNEG * TIMEGOAL 

i-l 

where m = the number of inspection tasks to be done. 
The right hand side of this constraint represents a goal in 
terms of time in which the manager wants inspections to be 
completed. The deviation from the goal which matters to the 
manager is captured by TPOS. 

(2-8) Z [ f(Xi) * C( error.) ] - ACCPOS + ACCNEG = ACCGOAL 

i = l 

where f(x^) = probability of inspector error as a 
function of time he spends on the 
inspection task, 

C(errori) = the cost of error^ in meters. 

The right hand side of this constraint represents a goal in 
terms of accuracy that the manager wants to be met. The 
deviation from the goal which matters to the manager is 
captured by ACCPOS. Deviations in one goal versus another are 
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encouraged through weights that are applied to the respective 
deviations in the achievement function. 

The desires of the decision maker are often 
preemptive: one tier of goals must be satisfied before 
another tier can be considered. Hillier and Lieberman 
describe preemptive goal programming and the handling of 
goals: 

Consider the case for preemptive goal programming. Such 
a case arises when one or more of the goals clearly is far 
more important than the others. Thus the initial focus 
should be on achieving as closely as possible these first 
priority goals. After we find an optimal solution with 
respect to the first priority goals, we can break any ties 
for the optimal solution by considering second priority 
goals. When we deal with goals on the same priority 
level, our approach is just like the one described for 
nonpreemptive goal programming. [Ref 5, p.245] 

Assume a number of independent visual inspection 
tasks which must be accomplished properly for an error-free 
product to come out of the facility. Assume that a cost, a 
quantitative measure relating to loss of customer 
satisfaction, can be associated with each error on a product 
that leaves the facility. If the manager can state a maximum 
tolerance cost, within which the confidence of the consumer 
is maintained, then the building of tiers can proceed in a 
manner such as this: 

TIER 1. 

Any inspection task covering a worker error which has a 
cost greater than the maximum tolerance cost should be 
considered first to maintain the consumer’s confidence. 
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TIER 2. 



If any time remains after the first tier inspection tasks 
are allotted their time then that remaining time can be 
applied to the second tier of goals. The second tier 
consists of any visual inspection tasks not addressed in 
tier one. 

If there is insufficient time to cover all tasks 
in the first tier, then goal programming is invoked to 

determine the allocation of time across those first tier 

visual inspection tasks. If sufficient time exists for the 
first tier tasks to receive maximum time then they are given 
their full allotment of time and goal programming is invoked 
to allocate time on the visual tasks in the second tier, 
d. Achievement Fvmction 

The achievement function takes the goals within 
one tier of the goal program and considers the deviations from 
each goal simultaneously. The deviations from goals are given 
weights by the decision maker in accordance with their 

relative importance to the organization. 

In the goal program for allocating visual 

inspection resources, the decision maker gives a weight to the 
time goal deviation and the accuracy goal deviation. The 
appropriate weight is multiplied by the deviation variable for 
each goal . The two weighted values are then summed to equal 
a total deviation for the problem. The achievement function 
minimizes this total deviation value. As the goal program 
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iterates, it trades off accuracy for time or time for accuracy 
until a solution is reached which best satisfies the two goals 
in light of their respective weights. 

The standard achievement function for allocating 
visual inspection time is given in Equation 2-9. 

(2-9) Minimize DEVIATION 

where DEVIATION = (TIMEWT * TPOS ) + (ACCWT * ACCPOS ) 
e. The Formulation 

The generic goal program to accomplish allocation 
of inspection time is shown in Figure 5. 

Achievement Function 

Minimize : 

DEVIATION = (TIMEWT * TPOS) + (ACCWT * ACCPOS). 

Goal Constraints 

Subject to: 

m 

Z Xi - TPOS + TNEG = TIMEGOAL 

i = l 

[ f(Xi) * C(errori) ] - ACCPOS + ACCNEG = ACCGOAL 

where m = the number of inspection tasks to be done. 
System Constraints 
< "^MAXi ' for i = 1 , 2 ,..., m 

Xi, ..., X„, TPOS, TNEG, ACCPOS, ACCNEG ^ 0. 

Figure 5 - Goal Program Formulation 
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F. SOFTWARE SUPPORT OF METHODOLOGY 

Suggested software support of the proposed methodology 
includes computer packages that swiftly perform a number of 
matrix operations and optimization. 

1. Matrix Operations 

A means to manage matrix operations is needed to 
transform the survey responses on worker error into useable 
data for the model. Some survey responses will be invalid due 
to internal inconsistencies. As discussed in Chapter II, B- 
4, eigenvalues are needed to determine the internal 
consistency in a group of judgments from a respondent. An 
eigenvalue solver or software which will allow the analyst to 
manipulate matrices to derive eigenvalues is necessary to 
permit a timely decision of whether to incorporate various 
respondent surveys into the model. 

A Programming Language (commonly known as APL ) is a 
software product that permits easy manipulation of matrices. 
The eigenvalue solver that accompanies most APL software 
packages is used on the illustrative problem of this thesis. 
APL is used in both Appendix A and Appendix D to derive the 
eigenvector from respondent surveys for the probabilities of 
crew error and to derive the maximum eigenvalue to determine 
respondent consistency. 
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2. An Optimizing Package 

A linear or nonlinear programming package is needed 
to optimize the objective function. There are 13 decision 
variables that simultaneously need to be considered. 

a. Nonlinear Programming Package 

A nonlinear programming package will permit a 
straight-forward solution to the model. Data which show the 
increase in inspector accuracy as a function of time is 
extracted from experimental studies which closely approximates 
the type of visual task being modeled. A curve fit is made 
to that data and a function is generated that approximates the 
curve fit. That function, if convex, can be included directly 
into the goal constraints for use by goal programming. The 
curves in all studies indicated in this thesis have indeed 
been convex . 

In the illustrative problem which accompanies this 
thesis the versatile and powerful Generalized Algebraic 
Modeling System (GAMS) package is used. GAMS allows for 
nonlinear programming. An example of the input and output 
which is produced with GAMS is shown in Appendix E. 

b. Linear Programming Package 

A linear programming package will permit a 
solution to the model. There is a minor degree of difficulty 
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in that the continuous curve generated by the curve fit in 
nonlinear progranuning must be broken up and made piecewise 
linear so that a linear programming package can solve the 
problem. 

The remainder of this thesis is the presentation 
of a problem to illustrate the methodology of allocating 
visual inspection resources in a production facility suggested 
in Chapter 2. 



52 



Ill . 



BACKGROUND TO AN ARTILLERY PROBLEM 



A. THE ARTILLERY ORGANIZATION 

The artillery fire support system consists of separate 
components which contribute to the goals of "rapid and 
accurate" fires. Depicted in Figure 6 are the components of 
the artillery system. 



OBSERVER FIRE DIRECTION CENTER 




Figure 6 - The Sub-system Components of the Artillery 
The observer, a Forward Observer ( FO ) or a RADAR section, 
spots a target. The Fire Direction Center (FDC) receives the 
target location information from the observer and translates 
that input into orienting data for the guns. The gun section 
prepares the ammunition, orients the weapon, and fires at the 
observer's target. Survey sections and meteorology sections 
support the gun section by giving topological surveyed grid 
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control and atmospheric meteorological information input into 
the system. 

The system components noted above are grouped into 
artillery organizations. Several gun sections and one FDC 
constitute a battery of artillery, commanded by a captain. 
Several batteries of artillery, observers, and surveyors are 
organized into a battalion of artillery, commanded by a 
lieutenant colonel. Several battalions and meteorologists 
compose a brigade of artillery, commanded by a colonel. 

B. ROLES OF THE PLAYERS. 

The focus of this thesis is the Gun Section. The Gun 
Section consists of the members depicted in Table 1. 



TABLE 1. MEMBERS OF A GUN SECTION 



position 


rank 


number 


yrs in 
service 


Chief 


E-6 


1 


5-15 


Gunner 


E-5 


1 


2-10 


Asst gunner 


E-4 


1 


1-8 


Ammo handlers 


E-1, 4 


3-7 


0-8 



The ammunition handlers prepare the shell that carries the 
munitions to the target, set the fuze that activates the shell 
at the appropriate point in its trajectory, and select the 
propellant charge that propels the shell out of the cannon and 
towards the target with the proper velocity. The assistant 



54 



gunner vertically orients the weapon tube to a designated 
position so as to obtain the trajectory needed to reach the 
range of the target. The gunner horizontally orients the 
weapon to a designated position to place the weapon tube on 
line with the target. The chief inspects the work of his 
subordinates to insure accuracy of settings and then commands 
the crew to fire the weapon. 

The FDC computes six elements of data that, when taken 
together, constitute the weapon aimpoint. Those six elements 
of data are the horizontal orientation setting, vertical 
orientation setting, shell type, fuze type, fuze setting, and 
amount of propellant. Firing the weapon at an aimpoint other 
than that directed by FDC is a gun crew error and is the 
responsibility of the chief. 

C. POSSIBLE ERRORS THAT A GUN CREW MAKES 

The four errors associated with the ammunition handlers 

are : 

1) improperly selecting the shell type, 

2) improperly selecting the fuze type, 

3) improper setting of the fuze, and 

4) improperly cutting the amount of propellant. 

The four errors associated with the assistant gunner and the 

vertical orientation of the weapon are: 

1 ) improperly setting the pre-designated four digit 
elevation number on the elevation counter. 
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2) improperly leveling the pitch bubble, 

3) improperly leveling the cross level bubble, and 

4) failing to have the proper two digit number in the 
piece correction window. 

Figure 7 shows the location of these four errors on the 
assistant gunner's mount of a towed howitzer. The four errors 
associated with the gunner and the horizontal orientation of 
the weapon are: 

1) improperly setting the pre-designated four digit 
number on the deflection counter, 

2) improperly leveling the sight mount bubbles, 

3 ) failing to have the proper two digit number in the 
piece correction window, and 

4 ) not traversing the weapon to acquire a proper 
sight picture through the panoramic telescope onto 
the aiming reference point. 

Figure 7 shows the location of these four errors on the 
gunner's mount of a towed howitzer. 

In addition to the above twelve errors that a chief must 
check prior to firing his weapon, there are checks he must 
make regarding the firing data his crew receives from the FDC. 
When firing into a clearly defined area for a prolonged period 
of time a chief is given a schematic that depicts the limits 
of the area into which he can safely fire. The schematic that 
depicts the limits for the safe fire of a weapon is called a 
"safety T" . The chief is required to check that the -settings 
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Figure 7 




- Potential Gunner and Assistant Gunner Errors 
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being made by his crew are within the horizontal orientation 
limits, vertical orientation limits, fuze setting limits, 
propellant limits, and ammunition constraints imposed by the 
schematic. In this way the chief insures that crewmen have 
not set grossly inaccurate data due to garbled communication 
from the FDC or from fellow crew members. As a minimum, the 
chief insures by means of the "safety T" that the round his 
section fires will fall within the limits of safe fire for 
munitions impact. 

D. THE GUN CHIEF'S TIME ALLOCATION PROBLEM 

The artillery's primary mission is to support the ground 
gaining maneuver forces with firepower to destroy the enemy 
or diminish his will to fight. The artillery accomplishes 
this mission by providing fast and accurate fires when called 
upon to do so [Ref. 43 p.l3]. Each component of the artillery 
fire support system is called on to contribute to the twin 
goals of fast and accurate fire. 

General Otis states, "(Artillery is) a widespread organism 
whose ultimate function is to deliver massive firepower 
exactly when and where the force commander wants it [Ref 30, 
p 29]." Fast and accurate artillery fire is what the maneuver 
force demands and what the artillery force must deliver. This 
thesis recognizes fast and accurate artillery fires as the 
preeminent consequences of interest (or measures of 
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effectiveness) for the entire artillery system. The final 
component, the gun section, shoots the projectile at the 
target. The crew of men that operate the gun is the focus of 
the illustrative problem of this thesis. 

The primary resources available during fire missions that 
directly influence gun crew firing accuracy are the gun 
chiefs. The commander employs the gun chiefs as his personal 
quality assurance inspectors [Ref 31, p.3-5] . These personnel 
effect accuracy by inspecting for gun crew errors prior to the 
firing of each round. There are sixteen distinct, mutually 
exclusive errors noted previously that the crew can make when 
firing the weapon. The limited resource that the inspector 
expends to influence accuracy is the time that he takes to 
inspect for errors. The inspector's time is limited because 
artillery fires must not only be accurate, but fast. 
Excessive time by the inspector detracts from promptness of 
fires . 

Current U.S. Army literature does not specify a method by 
which an inspector is to allocate his pre-fire inspection 
time. It is expected that a gun chief will use his artillery 
experience, knowledge of his crew, and intuition to decide 
where his time will best be spent between the various error- 
prone tasks he must cover. Intuition is inadequate in complex 
decision making, and this problem requires a complex decision 
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[Ref 3, p.l7]. Systems analysis and logic dictate the 
spending of limited error-stopping resources commensurate with 
the importance of the error. 

This illustrative problem presents an allocation of 
inspection time model for use by a commander. The model 
quickly and efficiently analyzes the importance of gun crew 
errors for an artillery unit by using subjective input from 
unit leaders garnered by means of surveys. The model then 
uses goal programming and suggests an allocation of pre-fire 
inspection time to reduce the probability of undesirable 
consequences due to errors. 

The alternative to allocation of resources by modeling is 
strict intuition. If intuition is failing in a unit as an 
effective means of allocating time then modeling offers an 
alternative. The alternative to modeling by means other than 
the quick techniques of the methodology in this thesis is to 
model using input from empirical observations. The use of 
empirical observations to generate error data is not viable. 

The error data needed are functions of training, 
experience, readiness, and type of cannon weapon system in the 
unit. These factors vary considerably between units. 
Consequently, information gleaned in one unit will have little 
use in another unit. The first three factors; training, 
experience, and readiness; can change considerably within the 
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same unit in a matter of weeks. Therefore, information 
gleaned in a unit may have little use in the same unit the 
following month. Gathering of unit error data through 
empirical observation for every artillery unit at frequent 
intervals is impractical, if not impossible. The modeling 
methodology must be as efficient as the situation is dynamic. 
The methodology presented in this thesis is useable by 
artillery units to allocate inspection time. 

E. ASSUMPTIONS 

Assumptions in this problem are in two categories. One 
set of assumptions isolates the effects of errors by the crew 
so that other off-setting or magnifying errors do not 
interfere with the analysis of the crew. The second set of 
assumptions facilitates modeling. 

1. Assumptions to Isolate the Problem 

The concern of this illustrative problem is the 
difference between the FDC's aimpoint (the target location) 
and the gun section’s aimpoint (the expected location of shell 
impact). That difference is the net effect of the error that 
the gun crew makes and the chief fails to catch. This error 
is measurable in terms of the number of meters by which the 
shell is expected to miss the target. 

To isolate the gun section regarding its contribution 
to fast and accurate artillery fire, it is necessary to make 
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some assumptions. For the purpose of this thesis the 
following simplifying assumptions regarding source of firing 
errors are made. 

a. The other artillery system components (besides the 
gun section) perform their respective tas)cs 
related to timely fire in the time that is 
expected of them. 

b. The aimpoint computed by FDC is the actual 
location of the target. 

1 ) No errors are made among the other 
artillery system components (FO, RADAR, 

FDC, Survey, Meteorology). 

2 ) The gun crews do not err when taking 
measurements for the FDC. Gun crews are 
periodically asked by the FDC to provide 
propellant temperatures and muzzle 
velocities to assist in a better computer 
solution. 

c. The aimpoint of the weapon is the precise impact 
point of the shell. 

1) Dispersion is disregarded. Artillery 
fire always experiences normal dispersion 
in terms of range probable error and 
deviation probable error about the 
targeted aimpoint. 

2 ) No errors are made by the battery 
leadership during the emplacement of the 
cannon weapon system. Prior to being set 
up to fire, the unit leadership emplaces 
the weapons. Errors occur during this 
emplacement which, if not corrected, will 
affect the accuracy of the artillery to 
follow. 

3 ) No errors are made when the gun crew 
performs routine serviceability checks of 
their weapon. Prior to weapon firing the . 
gun section performs tasks which can 
later affect accuracy. These tasks deal 
with such things as sight alignment. 
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checks to ensure equipment is within 
tolerance, and emplacement of aiming 
reference points. 

4) No errors occur in sub-systems directly 
related to the gun section. Events 
external to the gun crew which may 
negatively affect the accuracy of the 
artillery are defective fuzes, defective 
propellant, a round that fails to seat in 
the chamber when it is rammed, and an 
aiming reference point that is 
inadvertently moved. 

2. Assumptions to Facilitate Modeling 

a. Battery Homogeneity 

Gun sections are organized in batteries which 
train together and achieve some degree of homogeneity under 
their common commander. Among the duties of the commander is 
the establishing of training standards, standard operating 
procedures, and proficiency standards for the unit. It can 
be reasonably expected that standards are somewhat uniform 
across a battery. This uniformity allows for the gathering 
of unit data rather than individual section data. Within the 
unit, use of time and human error rates are basically 
homogenous. Because of battery homogeneity, data generated 
in one battery is of limited or no use in another battery. 

b. Competent Gun Chiefs 

A chief is capable. He is competent as a quality 
assurance inspector of his crew's work. Gun chiefs are 
required to go through thorough periodic testing to insure 



63 



their aptitude at finding all possible crew errors. It is 
assumed that a chief does not lose his ability to find 
errors. The chief's competence is such that he perceives his 
visual tasks as being relatively simple. Inspection failures 
are the result of random error, not incompetence. 

c. Gun Crew Errors are Independent of Each Other 
Crew errors are not totally independent. A 

poorly performing crewman is more likely to set a second 
error than a superb soldier is of setting his first. The 
interaction between errors is complex and unpredictable. It 
would be extremely difficult to model that interaction. The 
assumption of error independence is a rough approximation of 
reality to facilitate modeling. 

d. Type I versus Type II Error 

If the costs of identifying a good product as 
flawed (Type II error) is minimal compared to the costs of 
failing to identify a flawed product as flawed (Type I error) 
then Type II errors can be disregarded with little loss in 
model validity. This allows for simplification of the model 
and, in fact, is often the case in quality assurance 
inspection work. In the artillery gun section Type II error 
is commonly considered insignificant when compared to Type I 
error. No chief has ever been ruined by being overcautious 
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and calling a mission flawed when it was not. Many a chief 
has been ruined by failing to catch a flawed mission. 

e. Simplicity of Visual Inspection Tasks 

The visual inspection tasks required on a gun 
section are relatively simple to a trained cannoneer. The 
types of errors to be identified do not require a detailed 
search, exceptional visual acuity, or higher level cognitive 
skills . 

f. Effects of Errors are Cumulative 

The errors a gun section makes can often off-set 
each other such that the full effect of an error is not 
realized. This model assumes that when multiple errors occur 
on the same mission that their effects are cumulative along 
one dimension. Therefore, this is a "worst-case" analysis 
type model . 
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IV. MODELING THE ARTILLERY PROBLEM 



A. GENERAL CHARACTERISTICS OF THE MODEL 

The model to allocate a gun chief's inspection time is a 
goal program which suggests an amount of time a gun chief 
should spend on each of thirteen inspection tasks prior to 
every round fired from his weapon. The suggestion for time 
use is based on the following criteria for the tasks: 

1) The likelihood of crew error; 

2) The likelihood of inspector error; 

3) The potential costs of the fired error; 

4) The commanders' objectives. 

The method by which the data are gathered for the model takes 
into account the characteristics of the artillery weapon 
system and the level of training of the surveyed unit being 
analyzed . 

B. THE PROBABILITY OF GUN CREW ERROR 

Each of the 16 crew tasks which the chief inspects with 
his 13 visual inspection tasks has a distinct probability for 
error. Based on the battery homogeneity assumption, a 

probability for crew error applies across all the gun 
sections of a battery. As a result of his firsthand 
knowledge of the phenomenon, the gun chief has a strong 
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intuitive sense for the relative frequency of errors on the 
crew tasks that he inspects. 

Due to the battery homogeneity assumption and the fact 
that there are six or eight gun sections to a battery, there 
are seldom more than eight gun chiefs to give expert judgment 
on any crew error phenomenon under investigation. The paired 
comparison technique of estimating error is not viable unless 
there is a large number of respondents. Thus, the paired 
comparison technique is not a practical solution for the 
illustrative problem in this thesis. 

The model accepts the gun chief as an expert capable of 
rendering judgments on relative frequencies of crew errors. 
The model gathers this individual knowledge by means of a 
survey. Appendix G contains copies of the surveys 
administered to the gun chiefs. The survey forces the chief 
to make enough comparisons to allow for an internal 
consistency check on his judgments (Saaty [Ref 17]). Those 
judgments found to be sufficiently consistent are converted 
to probabilities using the indirect numerical estimation 
technique (Swain and Miller [Ref 1]) and averaged together 
(Von Winterfeldt [Ref 29]). The result is a deterministic 
point estimate of probability for each crew error in the 
battery. Appendix G shows the structure of the survey. 
Appendix C demonstrates checking the consistency of 
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individual surveys. Appendix A gives an example of 
converting paired comparison survey input into probabilities 
and the averaging of input from several experts. 

C. THE PROBABILITY OF INSPECTOR ERROR 

With 13 inspection tasks the gun chief checks the 16 
possible crew errors. Four of the crew errors, the four 
associated with the assistant gunner, are assessed by a 
single check that a chief makes using a precision tool called 
a gunner's quadrant. Within certain bounds inspector error 
is a function of time taken to inspect a crew task. Those 
bounds for a gun section are explored. 

1. A Function of Time 

For a fraction of a second at the start of an 
inspection task the inspector experiences no increase in 
accuracy performance due to time required for eyes to focus 
and the starting of the visual search. As the inspector 
takes more time his probability for successfully identifying 
an error, given that an error exists, increases at a 
decreasing rate (Rizzi, Buck, Anderson [Ref 25]). This 
increase in accuracy performance continues until the 
inspector spends 1.5 times the standard amount of time he 
would normally spend on that visual task (Schoonard, Gould, 
Miller [Ref 24] ) . 
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The experimental findings of Schoonard, Gould, and 
Miller are heavily incorporated to establish an upper bound 
for the productive use of time for a gun chief performing 
visual inspection tasks. Schoonard, Gould, and Miller's 
study [Ref 24] closely approximates the situation in which 
gun chiefs find themselves when inspecting for crew error. 
The study picked a technical task, the inspection of 
integrated silicon circuit chips, requiring trained 
inspectors. Each inspector had a minimum of six months 
experience inspecting for faulty chips. The actual visual 
search required the simultaneous searching for six or more 
different types of errors followed by a decision to accept or 
not accept the chip construction. The possible errors ranged 
from the simple to the complex in terms of the difficulty of 
discerning them during an inspection. The range of times 
that inspectors took to inspect a specific chip for errors 
was 2.7 to 5.4 seconds. In every aspect of the study noted 
above, Schoonard, Gould, and Miller come extremely close to 
the situation of a gun chief inspecting for a gun crew error. 

The model accepts that from the time the chief starts 
searching for errors until 1.5 times the standard time spent 
on that inspection task [Ref 24] the chief experiences an 
exponentially increasing probability of detecting an error 
[Ref 25]. At 1.5 times the standard time spent on an 
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inspection task a competent chief, with no distracting 
factors, detects 100% of errors (Drury [Ref 26]). If given 
more than 1.5 times the standard time for inspection the 
inspector experiences degradation in his performance (Kochhar 
[Ref 23] ) . 

Drury [Ref 26] and Smith and Barany [Ref 22 and 32] 
both develop a curve which represents the inspector's 
probability for accurate inspection results as a function of 
time. The curve increases at a decreasing rate, forming a 
gentle, convex curve. The curve is continuous from a minimum 
time which reflects the time it takes the eye and brain to 
start a visual search to a maximum time which reflects the 
limits of human vigilance for simple inspection tasks. The 
maximum time is assumed to be 1.5 times the standard time 
spent by the inspector on the task. The function chosen to 
represent the probability for inspector accuracy curve is: 

(4-1) Yi = ,/( Ti ) , i = 1 13 

Where Yi= probability for inspector accuracy on task i 

“ '^MlNi ) / ( "^MAXi “ '^MINi ) 

Tj= time chosen to spend on task^ 

This curve approximates the curves that Drury [Ref 26, p 264] 
and Smith and Barany [Ref 32, p 5] found in their work and 
which are depicted in Chapter II. C. 
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2. Establishing Standard Time Use on Visual Inspection 

Tasks 

To utilize the experimental findings that greatest 
inspector accuracy is achieved at 1.5 times the standard 
amount of time spent on a visual inspection task requires the 
generation of standard inspection times on all visual tasks. 
The standard time used to perform each visual inspection task 
could be gathered by empirical observation, but this 
illustrative problem uses the alternative of gathering the 
chief's knowledge of time utilization by means of a survey 
[Ref 14, p.6-1]. 

The chief has a strong sense for where his time is 
spent when he inspects for error. The chief performs paired 
comparisons of the amount of time spent on various inspection 
tasks. From the paired comparison data the model derives a 
point estimate for the time spent per inspection task by the 
gun chief. By averaging across all chiefs in a battery the 
model derives a battery estimate for chief's time spent per 
inspection task. See Appendix C for details on the 
structuring of a time use survey and of how a time use 
estimate is derived from actual data. 

3. Total Time to Perform All Inspection Tasks 

A limit on the total amount of time an inspector is 
permitted to perform all visual tasks may be determined by a 
commander. For general purposes of this problem, the Field 



Artillery School standard of 30 seconds is the amount of time 
permitted chiefs to inspect their crews prior to fire [Ref 
39 , p . 3 ] . 

D. THE PROBABILITY OF FIRING AN ERROR 

For an error to be fired the chief must be inspecting an 

improperly performed task and fail to identify the error. 

The probability that the crew performed a given task 

improperly is a point estimate derived by survey input. The 

probability that the chief fails to identify the error is a 

function of time. The model is shown in Figure 8. 

A technique in nonlinear programming for evaluating the 

curve in Figure 8 is shown in Equation 4-2. This technique 

explicitly models the curve shape shown in Figure 8. 

(4-2) Pji.ci = [ v/(l - Pci)^ * ^ + Pci i = 13; 

where T^= ( Tj - T„iNi ) / ~ ) , see Equation (4-1); 

Pii«ci = P(accuracyi) = probability for system accuracy 

on task^, 

P(errori) = 1 - P(accuracyi) = probability for system 

error on task^^. 

E. THE COST OF FIRING AN ERROR 

The costs for firing an artillery gun crew error range 
from the insignificant or the unnoticeable to the very 
significant. Some of the more important costs of a gun crew 
error being fired are: 
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1) loss of maneuver force confidence in their 
supporting artillery; 

2) the cost of the resupply effort to deliver more 
ammunition for that which was wasted; 

3) loss of destructiveness due to inaccuracy. 

The most important costs are directly related to the 
consequences of interest for the system. Timeliness and 
accuracy of fire are the consequences of interest to the 
artillery fire support system. 

Each of the 16 crew tasks has a different cost 
distribution associated with it in terms of inaccuracy (in 
meters), should the crewman err at performing his task. The 
model uses the PERT cost estimation technique to determine a 
cost distribution for each error [Ref 33] . Senior battery 
personnel that are well versed in both fire direction and gun 
crew operations determine the largest, the smallest, and the 
most likely error of each of the 16 error types. Using 
tabular firing tables [Ref 38] for the battery's weapon system 
and assuming a range at which targets are most likely engaged, 
a Beta distribution cost curve (in meters) is established for 
each error type [Ref 38 and Ref 33]. See Appendix B for 
details of the PERT cost estimation technique. 

The potential number of meters by which a crew error 
causes the round to miss a target is a good indicator of 
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Figure 8 - Probability of Firing an Error 
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either the amount of damage that can be expected on the target 
(small errors) or the amount of maneuver confidence that may 
be lost (large errors). 

F. MULTI -OBJECTIVE CRITERION OF THE COMMANDER 

The commander has objectives that he wants accomplished. 
Suppose, for instance, that a given commander, first and 
foremost, wants never to lose the confidence of the maneuver 
force he supports. Second, the commander wants to have 
effects on target. These hypothetical but realistic 
objectives of a commander are pursued in the remainder of this 
model to demonstrate the flexibility and real-world 
applicability of goal programming. 

1. Making a Goal out of First Objective 

For the purpose of this problem, assume that the loss 
of maneuver confidence in their supporting artillery is the 
most costly and damaging thing that can happen to the fire 
support system. There is good reason that this may well be 
true for any artillery unit. Loss of confidence results in 
hesitance on the part of maneuver to employ artillery and thus 
increases the chances of losing combat engagements due to lack 
of firepower. 

Confidence is difficult to quantify and place on some 
continuous scale. Therefore, a crew error cannot be directly 
linked to an amount of confidence lost by the maneuver force 
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towards its supporting artillery. It is intuitive that a miss 
by a large number of meters corresponds to a larger loss of 
confidence by the maneuver whereas a small miss may be 
mistaken as normal dispersion and thus goes unnoticed. 
Therefore potential number of meters by which a crew error 
causes the round to miss a target suffices as a surrogate for 
the amount of confidence lost. 

Below a certain distance threshold, maneuver 
confidence is not lost by artillery which misses its target. 
Artillery is an area fire weapon and it is expected by those 
who employ it that artillery will hit in the general vicinity 
for which it was called. Within that vicinity, or expected 
area of impact, the cost of crew error is just the cost of 
missing the target and the cost of wasted ammunition. 

The maneuver confidence radius represents the maximum 
distance from the target that a round can hit and still 
maintain the confidence of the maneuver force being supported. 
The commander's first goal is that his rounds must hit within 
the maneuver confidence radius of the targeted aimpoint. The 
commander needs to specify a maneuver confidence radius. 

2. Making a Goal out of the Second Objective 

The commander's second objective, getting effects on 
target, is clearly a function of both the accuracy of the fire 
and the timeliness of the fire. If accuracy is sought at the 
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excessive expense of time then the target will likely move and 
escape destruction. If timeliness is sought at the excessive 
expense of accuracy then the target will escape destruction 
due to rounds missing the aimpoint. A trade-off between time 
and accuracy is required to get effects on the target. The 
commander needs to specify the accuracy and timeliness goals. 

G. GOAL PROGRAM FORMULATION 

In the proposed multi-objective problem there are two 
objectives: avoid any loss of maneuver confidence and get 
effects on target. One goal is on the first tier: hit within 
the maneuver confidence radius of the target. Associated with 
the second objective is a tier of goals. On the second tier 
is an accuracy goal and a timeliness goal that the commander 
specifies based on his weapon system and the targets on which 
his weapon systems will most likely be employed. 

This two-tiered set of goals lends itself to multi- 
objective, preemptive goal programming. On the first tier is 
the commander's goal of keeping maneuver confidence. Once 
inspector time is allotted and this first goal is achieved 
then the second tier of goals is considered. 

1. Decision Variables 

In goal programming to achieve allocation of visual 
inspection resources, inspector time spent on checking the 
work of others is the decision variable ( X^, i=the number of 
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visual inspection tasks). The purpose of this goal 
programming is to achieve a most satisfactory allocation of 
inspector time, 

2. System Constraints 

The minimum and the maximum amounts of time that can 
be spent productively on each of the tasks are the two system 
constraints. These limits are based on the experimental 
findings of Drury [Ref 26] and Schoonard, Gould, and Miller 
[Ref 24] . Drury found a minimum amount of time required for 
an inspector's eyes, brain, and nervous system to function and 
start a visual search for error. Schoonard, Gould, and Miller 
found a maximum amount of time that an inspector spends at a 
visual inspection task before performance declines due to the 
limits of human vigilance. 

3. Goal Constraints 

There is not a single deterministic cost associated 
with each system error. There are three costs: a maximum, a 
minimum, and a most likely cost. Using the PERT cost 
estimation technique a Beta distribution curve is generated 
to fit these costs. Visual inspection tasks are related to 
accuracy goals through their individual beta cost functions. 

Visual inspection tasks fall into consideration in 
tier one if their minimum possible cost exceeds the maneuver 
confidence radius. Visual tasks fall into consideration in 
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tier two if their maximum possible cost exceeds the maneuver 
confidence radius. Tier three visual tasks have maximum 
possible costs below the maneuver confidence radius. 

Time is allotted by tier, thus reflecting the 
commander's priorities. The total amount of time required to 
fully satisfy (1.5 times the standard time) all visual tasks 
residing in a tier is compared to the total inspector time 
available. If time required is less than time available then 
all visual tasks in the tier are allotted a full complement 
of time and the model moves to the next tier of tasks. If 
time required to fully satisfy all visual tasks in a tier is 
greater than the time available then the goal program is run 
to determine allocation of inspection time among the visual 
tasks residing in the tier. 

Tasks within tiers are handled with the same goal 
constraints. These goal constraints are elastic, allowing 
some deviation if insufficient resources are available to 
satisfy all goals. 

The goal constraints are: 

(4-3) 

N 

tier 1 £ x„ - TPOS + TNEG = TIMEGOAL, 

n = l 

for all tasks n where minimum cost task„ > MCR, 

p 

tier 2 £ - TPOS + TNEG = TIMEGOALj 

p-i ^ 






for all remaining tasks (13 - N) where maximum cost of 
taskp > MCR, 

tier 3 ^ Xg - TPOS + TNEG = TIMEGOAL 3 

for all remaining tasks where (N+P+Q) =13. 

TIMEGOALi is the overall goal for time in which all tasks are 
to be completed. TIMEGOALj is the amount of time remaining 
from TIMEGOALj^ after the first tier tasks are satisfied. 
TIMEGOAL 3 is the amount of time remaining from TIMEGOAL^ after 
the first and second tier tasks are satisfied. 

(4-4) 

tier 1 E[P( error.) * MAXCOST„] - ACCPOS + ACCNEG = 0 

n = l 

for all tasks n where minimum cost task„ > MCR, 

tier 2 i;[P(error„) * MAXCOST„] - ACCPOS + ACCNEG = 0 
p=i P P 

for all remaining tasks (13 - N) where maximum cost of 
task„ > MCR, 

tier 3 ^[P(errorg) * MEANCOSTg] - ACCPOS + ACCNEG = 0 

for all remaining tasks where (N+P+Q) =13, 

where P(error^) = the probability of system error on task^ 

from Equation 4-2, 

MAXCOST^ = maximum cost of task^ (see Appendix B), 
MEANCOSTj = mean cost of task^ (see Appendix B). 

MCR = maneuver confidence values 
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The right hand side of the constraint represents a goal in 
terms of accuracy ( in meters ) that the commander wants to be 
met . 

4 . Achievement Function 

The commander assigns a weight to the time goal 
deviation and the accuracy goal deviation in the model. The 
commander must weight the deviations from each goal with 
respect to their relative importance to the fire support 
system. The purpose of the goal program is to simultaneously 
minimize deviations from goals within tiers. 

The commander is generally aware of the type of combat 
in which he will be employed and the type of targets on which 
he will be called to deliver his fires. Most targets of 
opportunity have the potential to move at some rate. That 
rate (meters per second) provides the commander the knowledge 
he needs to make a trade-off between timeliness of fire and 
accuracy of fire. The commander expresses this trade-off by 
the weight he assigns to the respective goal deviations. 

The appropriate weight is multiplied times the 
deviation variable for each goal. The two weighted values are 
then summed to equal a total deviation for the problem. The 
achievement function minimizes this total deviation value. 
As the goal program iterates, it trades off accuracy for time 
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or time for accuracy until a solution is reached which best 
satisfies the two goals in light of their respective weights. 
The achievement function is given in Equation 4-5. 

(4-5) 

Minimize : 

DEVIATION = (TIMEWT * TPOS ) + (ACCWT * ACCPOS ) . 
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V. 



APPLICATION TO ACTIVE ARMY UNIT 



A. CURRENT ALLOCATION OF INSPECTOR TIME 

The competent chief will attempt to at least look at every 
one of the 16 error-prone tasks accomplished by his crew. The 
chief’s underlying belief is that he is a good inspector and 
that if he looks at the possible error he will likely catch 
the error if it is present. 

The following are the keys to modeling the current 
allocation of chief's time on the pre-fire inspection: 

1) The gun chief spends at least the theoretically 
derived minimum time for a visual inspection task 
on every task. 

2 ) The gun chief spends additional time on those 
tasks that he knows are more frequently in error 
than others in the attempt to catch as many errors 
as he can. 

3) The chief does not trade time for accuracy. He 
shoots within the time required of him ( 30 
seconds) and checks as best he can (using rule 1 
and 2) during that period of time. 

The model proposed is simple, yet it mirrors the chief's 
decision making process quite well. The idea behind the 
current use of inspector time is summed up by the one goal : 
stop all errors within the time constraint to shoot. A GAMS 
program which reflects these priorities and uses the 
probabilities for error developed in this thesis is shown in 
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Appendix E. The allotment of inspector time resulting from 
this model is shown in TABLE 2. 



B. ALTERNATIVE ALLOCATION OF INSPECTOR TIME 

The following are the keys to modeling the alternative 
allocation of chief's time on the pre-fire inspection: 

1 ) Visual inspection tasks covering crew errors with 
minimum possible costs that exceed the maneuver 
confidence radius must be considered first for 
allocation of inspection time. The maximum 
possible costs of the errors are considered when 
the allocation of time is made. 

2 ) Visual inspection tasks covering crew errors with 
maximum possible costs that exceed the maneuver 
confidence radius are considered next for 
allocation of inspection time. The maximum costs 
of the second tier errors are considered when the 
allocation is made. 

3 ) Visual inspection tasks covering the remaining 
errors are considered last for allocation of 
inspection time. The expected costs of the third 
tier errors are considered when the allocation is 
made . 

The alternative model incorporates this information before 
allocating inspector time. In this example of an alternative 
plan for the use of the inspector's time the commander's goal 
is to stop costly errors. The value that drives the goal, in 
this instance, is maneuver force confidence and target 
destruction. The results will be different than the current 
allocation of inspector time which was driven by the 
inspector's goal of stopping all errors. 
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Appendix E gives the GAMS computer runs of both the 
current and alternative allocation of inspection time by an 
active army unit surveyed at Fort Ord, California. TABLE 2 



shows the 


different allotments 


of inspector 


time 


per 


inspection 


task that 


resulted from 


the two model 


runs. 




TABLE 2 . 


CURRENT VS 


ALTERNATIVE ALLOTMENT OF INSPECTOR 


TIME 


Inspector ' 


Task 


Current 


Alternative 


Difference 






Allotment 


Allotment 


Col 2 - 


Coll 


G1 




2.24 


2.80 


+ 


.56 


G2 




2.56 


1.05 


- 


1.51 


G3 




1.00 


1.00 




0 


G4 




1.60 


1.46 


- 


.14 


AG 




7.21 


8.90 


+ 


1.69 


A1 




1.80 


1.80 




0 


A2 




1.78 


.75 


- 


1.03 


A3 




1.70 


1.27 


- 


.43 


A4 




3.40 


3.40 




0 


SI 




1.14 


3.00 


+ 


1.86 


S2 




2.00 


2.00 




0 


S3 




2.15 


3.50 


+ 


1.35 


S4 




1.40 


1.23 


- 


.17 
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VI. CONCLUSIONS AND RECOMMENDATIONS 



A. CONCLUSIONS 

Visual inspection to identify worker error often involves 
trained inspectors to perforin a task requiring technical 
expertise. Human accuracy at visual inspection tasks is 
generally a function of the time taken to inspect. Inspector 
accuracy at positively identifying existing errors increases 
at a decreasing rate as a function of time. There are lower 
and upper bounds on the time axis creating a range of time 
over which this increasing inspector accuracy is a continuous 
variable. Those bounds have been explored experimentally and 
are available for use by systems analysts. 

If the possible human errors on a work site can be 
identified then expert judgments of the personnel involved in 
the actual operational work can be used to derive the 
probability for worker error on each task. In the same way, 
the standard time an inspector spends on the inspection of the 
task can be derived. Costs of system failures resulting from 
task error can be estimated by the management. 

The expected cost of doing business can be written as a 
function of inspector time spent on each component task in the 
system. If the decision maker can establish goals for the 
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system cost of a product and the acceptable amount of time for 
an inspector to spend on the product, then goal programming 
provides a method for the organization to optimize the use of 
inspector time. 

B. POSSIBLE APPLICATIONS OF METHODOLOGY 

Small-scale operations that employ visual inspectors to 
check the work of production personnel are prime candidates 
for use of this methodology. The small-scale operation is 
emphasized because it is most likely one in which a formal 
system analysis has not been conducted to evaluate optimum 
times for visual inspection or optimal rates by which 
production must occur. 

An abbreviated but effective systems analysis approach to 
the proper utilization of inspector time requires only a few 
probability estimations and a few cost estimations. The fact 
that the personnel on the weapon systems can provide the 
probability of error information and the unit leaders can 
provide the cost of error information means that individual 
units can apply systems analysis to better achieve their 
goals . 

The pace at which the product is presented to the visual 
inspector has everything to do with whether the visual 
inspection will have the desired effect of the management or 
not. In this case a little bit of math goes a long way. By 
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using eigenvalues to establish validity of expert judgments, 
the model considers only those respondents whose judgments are 
consistent- Some adding, multiplying, and dividing achieves 
probabilities for each crew error. The cost estimation 
technique uses a simple beta distribution to model the cost 
distribution for each error. The model requires only three 
rough estimates on the part of a knowledgeable manager. The 
mean of the cost distribution is achieved by some subtraction 
and division. The probability of inspector error and the 
final cost of doing business is determined by goal 
programming. The input for the goal constraints are obtained 
from the decision maker. The result is a recommendation, by 
task, for the use of inspector time such that the organization 
goals are met as closely as possible. 

C. AREAS FOR FUTURE RESEARCH 

A curve fit on some experimental results done in the 
1970 ’s on the relationship between time spent on a visual 
inspection task and the accuracy of the inspector at 
positively identifying existing errors yielded a curve of 

P(IA) = , 

where P(IA) is the probability of inspector accuracy and 
T is the time the inspector takes to inspect. 

This is basically the curve used for all visual inspection 

tasks in the illustrative problem used in this thesis. An 
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area for future research is classifying different curves with 
different levels of difficulty of the visual inspection. It 
appears from the experimental results that with an increase 
in difficulty in the visual task, the rate of improvement in 
accuracy changes from increasing at a decreasing rate to 
increasing at a slower, more constant rate. If error-prone 
visual inspection tasks can be rated according to difficulty 
and different accuracy curves can be generated for difficulty 
levels then the model may be more valid. 

Another area that may yield interesting results is 
weighing of the various expert judgments using some criteria. 
Suggested criteria that are readily discernable from the 
respondents are time in service, time in the job, rank, and 
skill qualification test score. Weighted judgments may yield 
probabilities that are more reliable or verifiable than giving 
each respondent equal weight in their input to unit error 
probabilities. It may be determined that certain 
qualifications are significantly more important than others 
in the development of a gun chief. 

The model developed is entirely deterministic. The model 
develops probability and time estimates from several sources. 
From the several estimates a distribution could be established 
with a standard deviation on the mean point estimate. The 
cost estimate procedure using PERT methods generates a Beta 
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distribution for costs of each error. A simulation could be 



developed incorporating the distributions from all the data 
collected in this model which would yield a more robust 
solution. By running the simulation a number of times, 
confidence intervals for the various errors could be 
generated. 

This thesis put time and probability phenomena on an 
interval scale of measurement by using paired comparisons and 
indirect numerical estimation methods. The anchoring of that 
interval scale to a ratio scale was done by direct numerical 
estimation of some single phenomenon by the respondents. 
Direct numerical estimation has flaws as a technique for 
gathering data from operational personnel. An area for future 
research would be to design an experiment and generate actual 
values from empirical observation for the estimates which 
anchor the interval scaled values to a ratio scale. 
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APPENDIX A 



GENERATING PROBABILITIES FOR CREW ERROR 

The steps to generate probabilities for crew error using 
expert judgment are given below with a numeric example taken 
from survey input out of an active army unit. 

1. Create a fault tree. See Figure 9. Worker errors 
appear in the lowest, or first level. Errors are grouped by 
worker, by work station, or by some other criteria that is 
easily discernable by the inspector. Those criteria form a 
second level of the fault tree and fall under the general 
category of "error". "Error" and "no error" form the third 
level of the tree. 

2. Gather survey responses from inspectors. Inspectors 
are called on to relay their firsthand knowledge of relative 
frequencies of errors. Inspectors do pairwise comparisons as 
to relative frequencies of errors within criteria groups. 
When all groups have been judged then pairwise comparisons as 
to relative frequencies are performed on the groups 
themselves. Finally a direct numerical estimation is made by 
the inspector as to the frequency of missions that occur which 
have any error. A sample survey used in an artillery unit is 
included in Appendix G. 
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3. Turn survey responses into a reciprocal matrix. Ones 
appear on the main diagonal. Survey responses of relative 
frequencies fill the upper triangular matrix and their 
reciprocals fill the corresponding positions in the lower 
triangular matrix. See Table 2. 

4. Derive the maximum eigenvalue and the eigenvector for 
the reciprocal matrix. The eigenvector is the relative 



frequency 


of 


each error 


from the 


perspective of 


one 


respondent . 


See Table 3. 


The maximum 


eigenvalue is used in 


Appendix C 


to 


derive a 


measure of 


consistency for 


the 


respondent . 












5 . Turn 


frequency 


scores into 


probabilities . 


By 


normalizing 


the 


eigenvector 


(making its components sum to 


one ) 



the probability for each error is derived, given that some 
error within that group of errors did occur. See Table 4. 

6. Across all consistent responses within a unit (use 

method in Appendix C to establish consistency), average the 
probability of each error type to make a unit probability. 
See Table 5. Fill those unit average probabilities back on 
the fault tree in the P(j), P(k), P(l), P(m), and P(n) 

positions. See Figure 10. 

7. Take the direct numerical estimates of the probability 
for a fire mission containing an error of any sort. Average 
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them for a unit probability of fire mission error. Fill this 
value in on the fault tree in the P(ii) position. 

8. Use the multiplicative rule down each branch of the 
fault tree to derive a probability for each type of error on 
the first level. 



TABLE 3. RECIPROCAL MATRIX FOR GUNNER ERROR FREQUENCY. 
( taken from a respondent’s survey on gunner errors ) 
relative | 



frequency 
of errors 


1 


error 

2 


type--- 

3 


4 


error 1 


1.00 


1.00 


3.00 


3.00 


type 2 


1.00 


1.00 


2.00 


2.00 


3 


0.33 


0.50 


1.00 


1.00 


4 


0.33 


0.50 


1.00 


1.00 



TABLE 4. EIGENVECTOR FOR THE RECIPROCAL MATRIX 

error type 1 .718 
error type 2 . 587 
error type 3 .263 
error type 4 .263 



TABLE 5. NORMALIZED EIGENVECTOR 

Eigenvector normalized to make probabilities: 

error type 1 .39 

error type 2 .33 

error type 3 . 14 

error type 4 . 14 
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TABLE 6. AVERAGE OF RESPONDENT'S ESTIMATES 



Average of five consistent 
respondents' probabilities. 

error type 1 .30 
error type 2 .36 
error type 3 .18 
error type 4 .16 
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Figure 9 - Probability of Crew Error Fault Tree 





Figure 10 - Probability of Crew Error Data 

96 



APPENDIX B. 



GENERATING COST ESTIMATES FOR ERROR 

Crews make errors and inspectors fail to identify those 
errors. The costs that result are measured in meters by which 
the error is expected to cause the round to miss the target. 
Each error has a maximum amount that it can cost the system. 
That maximum is affected by such things as unit standard 
operating procedures and unit safety measures which may 
preclude an extreme error. One example is the M102 howitzer. 
When traversed, the entire metal structure pivots on a ball 
mounted to a base plate. If traverse limits exist for firing 
safely then it is a common technique to drive metal stakes in 
the ground abutting the metal structure when it is lined up 
to fire on the left and right traverse limits thus physically 
limiting the weapon to firing within safe traverse limits. 
The result is an administrative method of precluding gross 
firing errors to the left or right of the target area. This 
naturally reduces the possible cost of traverse, or deflection 
error . 

Because there are many procedures in the field on five 
different cannon weapon systems for precluding various firing 
errors, there exist no minimum and maximum costs for crew 
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errors that apply across the army. It is within the means of 
senior personnel in each unit, however, to consider the unit's 
procedures and weapon system to establish a minimum, maximum, 
and modal value for each error type. The steps to take in 
establishing the key values for each error type are reviewed. 

1. Assume an average range at which targets are 
engaged. Assume a charge with which those targets 
are engaged. 

2. Extract information from the tabular firing tables 
( TFTs ) for the respective range and charge 
assumed. Information to be extracted is the 
effect (measured in meters) of a round that, when 
fired, is off by: 

a) 1 mil in elevation 

b) 1 mil in deflection 

c) 1 square weight 

d) 1 charge of propellant 

e) 0.1 second on time fuze setting 

3. Consider unit procedures that effectively limit 
the size of errors that can possibly be fired. 
Determine a maximum amount of error that can be 
set by a worker in terms of the items listed above 
(mils, square weights, charges, seconds) and 
convert that amount to meters on the ground. 

4. Given that he must set an error, determine a 
minimum amount of error that a crewman can 
possibly set. For example, a crewman can set no 
less than a one charge error without performing 
perfectly in that regard. So the minimum possible 
error for charge is that number of meters 
corresponding to a one charge error. 

5. Given that he must set an error, determine a most 
likely error that can be fired. 
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In the case of a 105 nun artillery unit at Fort Ord, 
California, the following assumptions were made about the 
potential costs of the various inspection task errors. 

1) Safety Ts used, 

2) Safety stakes employed, 

3) Dog-legged safety box in impact area, 

4) Charge 5 propellant required, 

5) Range of target: 6,000 meters. 

The following matrix was developed from the expertise of 
several unit leaders. 



TABLE 7. COST OF FIRING A CREW ERROR (IN METERS) 



Inspection 

Task 


estimated from 
experience 


computed assuming 
a Beta Distribution 


Min 

Cost 


Modal 

Cost 


Max 

Cost 


Mean 

Cost 


Std 

Devn 


G1 


0 


30 


2400 


420 


400 


G2 


0 


10 


200 


40 


33 


G3 


0 


60 


2400 


440 


400 


G4 


0 


15 


400 


74 


66 


AG 


0 


20 


2600 


447 


433 


A1 


950 


1200 


2400 


1358 


242 


A2 


25 


30 


100 


41 


12 


A3 


0 


20 


120 


33 


20 


A4 


0 


100 


2500 


483 


417 


SI 


10 


120 


2400 


481 


398 


S2 


15 


130 


2000 


422 


330 


S3 


25 


260 


2500 


594 


412 


S4 


0 


100 


150 


92 


25 
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APPENDIX C. 



GENERATING TIMES FOR INSPECTION TASKS 

Two methods are suggested by the NRC for generating 
standard times for inspection tasks to be completed. One is 
by measuring empirical observations and the other is by expert 
judgment. Making empirical observations is the preferred 
method and is the one suggested if a unit has the time and 
resources to devote to an experiment to measure a number of 
observations . 

The steps to generate standard times for inspectors to 
inspect for crew error (using expert judgment) are given below 
with a numeric example taken from survey input out of an 
active army unit. 

1. Create a fault tree of crew tasks. See Figure 10. 
Worker errors should appear in the lowest, or first level. 
Errors are grouped by worker, by work station, or some other 
criteria that is easily discernable by the inspector. Those 
criteria form the second level of the fault tree. 

2. Gather survey responses from inspectors. Inspectors 
are called on to relay their firsthand knowledge of relative 
lengths of time required to inspect for various errors. 
Inspectors do pairwise comparisons as to relative lengths of 
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time to inspect for errors within criteria groups. When all 
groups have been judged then pairwise comparisons as to 
relative lengths of time required to inspect the groups of 
errors themselves are done. Finally a direct numerical 
estimation or an empirical measurement is made by management 
of the time required to inspect for one task. 

3. Using the one task as a reference point, all other 
tasks are given a length of time for inspection. When an 
estimate exists for all tasks then a unit average is taken and 
that becomes the standard inspection time for the unit. 
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APPENDIX D 



CONSISTENCY CHECKS ON SUBJECTIVE EVALUATIONS 
The method for checking internal consistency of a set of 
pairwise comparisons made by a respondent is reviewed. This 
particularly applies to comparisons of relative frequency of 
various events, such as event A is three times more likely 
that event C. 

1. Follow steps 1 through 4 of Appendix A to derive a 
maximum eigenvalue for the respondent's reciprocal matrix. 

2. Generate a consistency index (Cl) for the respondent's 
matrix by 

[ ] / [ N-1 ] = Cl. 

3. Generate a quantity of random reciprocal matrices of the 
same dimension and containing the same range of values as was 
possible to the respondents on the survey they were 
administered. The main diagonal of these matrices are ones. 
The upper triangular cells of the matrices are filled with 
values picked at random from the possible range. Complete the 
lower triangular cells with the corresponding reciprocal 
values of the upper triangular cells. Compute the maximum 
eigenvalue for each matrix and average all of the maximum 
eigenvalues together. The result is an estimated maximum 
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eigenvalue for a respondent who gives totally random responses 
to the original survey. 

4. Generate a consistency index for the random matrix 
[Cl 

random 3 * 

[ ^MAXrandom ~N]/[N~1] — 

5. Generate a consistency ratio by 

( Cl respondent ) / [Cljandom) ” CR . 

If CR is less than or equal to .10 then there is a sufficient 
level of internal consistency in the respondent’s judgments 
to consider them for analysis work [Ref 17, p.83]. 

Step number three above requires some effort and 
significant computer resources on the part of an analyst. 
This step is made considerably easier if the analyst prepares 
the respondent's survey so that only values between one and 
nine (and their reciprocals) can be selected by the 
respondent. The range of one to nine recurs frequently in 
human judgments. For this range of values Saaty [Ref 17] 
generated consistency indices (Cl) for random reciprocal 



matrices from 


size 


(3 X 3) to 


(10 X 


10) . 


Table 7 


shows 


the 


random 


consistency 


indices for 


matrices up 


to (10 


X 10). 






TABLE 


8. Random Consistency 


Indices (CI„„aom) 




N X N 


2 


3 


4 5 


6 


7 


8 


9 


10 


PT 

random 


0.00 


0.58 


0.90 1.12 


1.24 


1.32 


1.41 


1.45 1 


.49 
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APPENDIX E 



USING GAMS FOR GOAL PROGRAMMING 

GAMS is used to perform the allocation of inspection time 
by the Current Model and the Alternative Model. 

1 . The Current Model 

In the Current Model the chief is aware of only the 
inspection tasks he must perform, the relative frequency by 
which the tasks are successful at discovering crew error, and 
the relative amount of time it takes to perform the visual 
tasks. Given only that much information the chief's strategy 
is to stop as many crew mistakes as he can in the time 
allowed. The assumption is made that the chief's intuition 
works very well with the knowledge that he has and that he 
does accomplish his objective of minimizing the probability 
for a crew mistake. It is reemphasized that the chief does 
not consider the relative magnitude of the costs of the 
mistakes when he currently performs the pre-fire inspection. 
The GAMS program for currently allocating inspection time is 
labeled "Current Model". 

2. The Alternative Model 

The Alternative Model takes what the chief knows in 
the Current Model and then takes into account the costs of the 
errors. The purpose of the Alternative Model is to not make 
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costly mistakes while meeting the commander's guidance for 
fast and accurate fires. In the first tier of the preemptive 
goal program any inspector error which has a minimum cost 
greater than the commander's maneuver confidence radius (600 
meters for the illustrative problem) is considered. Any first 
tier error will certainly have a huge negative effect on the 
artillery system. Every possibility for first tier error must 
be precluded. This is done by placing a large weight (BPWT) 
on any positive deviation from the accuracy goal. 

In the second tier of the preemptive goal program any 
inspector error which has a maximum cost greater than the 
commander's maneuver confidence radius is considered. Any 
second tier error potentially has a huge negative effect on 
the artillery system. After first tier errors are precluded 
then every possibility for a second tier error is precluded. 
Again, this is done by placing a large weight (BPWT) on any 
positive deviation from the accuracy goal. 

In the third tier of the preemptive goal program all 
remaining inspector errors are considered. Time that remains 
after the first and second tier errors are precluded is now 
allotted to the third tier inspection tasks. The commander 
now weights timeliness of fire and accuracy of fire relative 
to each other. In the illustrative problem the commander 
chooses to weight them equally (TIMEWT = BPWT = 1.0). 
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An asterisk in the first column of the GAMS computer 
input is a line that the computer program does not read. In 
the Alternative Model (2) asterisks are used in the PARAMETERS 
part of the program so that only the inspector tasks for the 
particular tier being considered are entered into the program. 
The GAMS input for the Alternative Model's first tier (2) is 
given. The second tier and third tier are the same programs 
but different tasks are allowed or disallowed by using 
asterisks. In each of the three tiers the SCALARS change to 
reflect the tier. The first tier starts with 30 seconds of 
time available (RESPTIME). The weights for time deviation 
(TIMEWT) and accuracy deviation (BPWT) are 1:600. The second 
tier starts with 28.2 seconds of time available. The weights 
for time and accuracy deviation remain 1:600. The third tier 
starts with 3.6 seconds of time available. The weights for 
time and accuracy deviation become 1:1. 

3 . The Output 

a) The Current Model output is shown first. The 
allocation of time by the Current Model for each 
inspection task is shown in the column labeled 
LEVEL between the LOWER and UPPER limits. This 
appears near the bottom of the page of GAMS 
computer output. 

b) The Alternative Model output is on three pages. 

At the bottom of the first page is the allocation 
of time for those tasks identified as in the first 
tier ( task A1 ) . 

c) At the bottom of the second page is the allocation 
of time for those tasks identified as in the 
second tier (tasks Gl, G3, AG, A4, SI, S2, S3). 
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d) At the bottom of the third page is the allocation 
of time for those tasks identified as in the third 
tier (tasks G2, G4, A2, A3, S4). 
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1. The Current Model. 

a - 



GPNLP2 



CURRENT MODEL 

3 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

^ SET 

5 I thirteen inspection tasks in four categories 

6 /G1,G2,G3,G^,AG,A1,A2,A3,A<<,S1,S2,S3,S^/; 

7 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

8 

9 PARAMETERS 

10 

11 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

12 X these min times are theoretical; based on experiments of min time for 
15 X eyes to focus and mind to grasp what eyes are seeing 

lA 

15 MIN(I) min time possible on task i- theoretical seconds 

16 



17 


/G1 


.7 


18 


G2 


.7 


19 


G3 


.7 


20 


G^ 


.7 


21 


AG 


2.0 


22 


A1 


.7 


23 


A2 


.7 




A3 


.7 


25 


A^ 


.7 


26 


SI 


.7 


27 


S2 


.7 


28 


S3 


.7 


29 


S^ 


.7 



30 

31 XX¥XXXXXXXXXXXXXXXXXXX¥XXXXX¥¥¥XXXXyXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

32 X these times incorporate theoretical 1.5 times max values of standard 

33 X time taken by gun chiefs to perform the task. This is time needed 
3^ X to insure (100^) that task is done correctly. 

35 

36 MAX(I) max time spend on task i “ from paired comparisons 

37 

38 



39 


/G1 


2.8 


^0 


G2 


3.2 


A1 


G3 


1.0 


^2 


GA 


1.6 


A3 


AG 


8.9 


AA 


A1 


1.8 


A5 


A2 


A. 8 


A6 


A3 


1.7 


A7 


AA 


3. A 


A8 


SI 


3.0 


A9 


S2 


2.0 


50 


S3 


3.5 


51 


SA 


1 .A/ 



52 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

53 NEPROBCI) probability of no crew error task i “ detm by INE method 
5^ 



55 


/G1 


.992200 


56 


G2 


.9906A0 


57 


G3 


- .995320 


58 


GA 


.9958A0 
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The Current Model (continued). 



GPNLP2 



59 


AG 


.97A000 


60 


A1 


.9888A6 


61 


A2 


.99087^^ 


62 


A3 


.987325 


63 


A^ 


.982255 


6^ 


SI 


.995632 


65 


S2 


.99399^ 


66 


S3 


.99126A 


67 


SA 


.991810/ 


68 






69 




70 


EPROB(I) 


probabi 1 i ty 


71 






72 


/G1 


.007800 


73 


G2 


.009360 


7^ 


G3 


.00A680 


75 


GA 


.00A160 


76 


AG 


.026000 


77 


A1 


.01115A 


78 


A2 


.009126 


79 


A3 


. 012675 


80 


AA 


.0177A5 


81 


SI 


.00A368 


82 


S2 


.006006 


83 


S3 


.008736 


8A 


SA 


.008190/ 



on task i 



detm by INE method 



85 

86 

87 

88 

89 

90 

91 

92 

93 
9 ^ 

95 

96 

97 

98 

99 
100 
101 
102 
103 
10 ^ 

105 

106 

107 

108 

109 

110 
111 
112 
113 
in 



X A chief does not consider the costs of various errors as he determines 
X where he will spend his time during the inspection. His sole objective 
* is to stop as many errors as possible in the time alotted 

SCALARS 

RESPTIME total time allowed for guns to shoot / 30.0/ 

VARIABLES 

X(I) time to be spent inspecting each task in battery 

DEVIATION deviation from the objective function; 

POSITIVE VARIABLES 
X(I); 

KXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXKXXXXXXXXXXXXXXXKXXXXXXXX 

EQUATIONS 

OBJDEF achievement function 
TLIMIT time limit; 

X this equation sums the probability for system error (which is a 

function of the decision variable) across all the visual inspection 



tasks AND assigns that sum a variable name of DEVIATION. 
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The Current Model (continued) 



III 



GPNLP2 



115 X MINIMIZE 

116 OBJDEF.. DEVIATION =E= 

117 ( SUM ( I, 

118 ( 1 - SQRT( (SQR(EPROBd))) K (X( I )-MIN( I ) ) / (MAX( I )-MIN( I) ) ) 

119 - NEPROB(I) ) ) ); 

120 

122 X this constraint limits time spent by the inspector across all visual 

123 X inspection tasks to less than or equal to the commander’s goal. 

12^ 

125 X SUBJECT TO 

126 TLIMIT.. SUM ( I, X(I) ) =1= RESPTIME ; 

127 

128 X ADDITIONAL CONSTRAINTS involving min and max times allowed. These 

129 X keep x in the feasible region as problem iterates^ otherwise stops 

130 

131 X.LO(I) = MINCDXI.OI; 

132 X.UP(I) = MAX(I); 

133 

13<^ X If region is convex# then no matter where program starts# the 

135 X solution will be the same. This forces program to start internally 

136 X on the convex region because funny things happen on the edges 

137 

138 X.L(I) = MIN(I)X1.02 

139 ; 

1^0 MODEL GPNLP2 /ALL/; 

1^1 SOLVE GPNLP2 USING NLP MINIMIZING DEVIATION; 
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2. The Alternative Model. 



a 



GPNLP2 



5 

6 

7 

8 
9 

10 

11 

12 

15 

1^ 

15 

16 

17 

18 

19 

20 
21 
22 
23 

25 

26 

27 

28 

29 

30 

31 

32 

33 
3 ^ 

35 

36 

37 

38 

39 
<50 



ALTERNATIVE MODEL (first tier) 

3 X THIS MODEL PERFORMS FIRST TIER OF PREEMPTIVE GP (ON THOSE TASKS WITH 
X MIN COST GREATER THAN THE MANEUVER CONFIDENCE RADIUS = 600m) 



5<5 

55 

56 

57 

58 



Xresp time ballpark ti.wt, bp.wt 



30 



1 



/ 600 / 



XX)(XXXXXXXXXXXXXXXXXXXXXXXXX)000(XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 

SET 

I thirteen inspection tasks in four categories 
/ 

X Gl # 

X G2, 

X G3, 

X G^, 

X AG, 

A1 

X , 

X 
X 
X 
X 
X 
X 
X 
X 



A2, 

A3, 

A<5, 

51, 

52, 
S3 



S^ 



/; 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

PARAMETERS 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
X these min times are theoretical; based on experiments of min time for 
X eyes to focus and mind to grasp what eyes are seeing 

MIN(I) min time possible on task i- theoretical seconds 



<51 


X 


Gl 


.7 


<52 


X 


G2 


.7 


<53 


X 


G3 


.7 


<5 <5 


X 


G<5 


.7 


<55 


X 


AG 


2.0 


<56 




A1 


.7 


<57 


X 


A2 


.7 


<58 


X 


A3 


.7 


<59 


X 


A<5 


.7 


50 


X 


SI 


.7 


51 


X 


S2 


,7 


52 


X 


S3 


.7 


53 


X 


S<5 


.7 



/ 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
X these timeSw incorpora te theoretical 1.5 times max values of "normal” 
X time taken by gun chiefs to perform the task. This is time needed 
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The Alternative Model (continued) 



GPNLP2 



59 


X 


to insuro 


(lOOX) that task is dona correctly. 


60 








61 




MAX(I) max 


time spend on task i ~ from paired comparisons 


62 






- 


65 








6^ 




/ 




65 


X 


G1 


2.8 


66 


X 


G2 


5.2 


67 


X 


G5 


1.0 


68 


X 


G^ 


1.6 


69 


X 


AG 


8.9 


70 




A1 


1.8 


71 


X 


A2 


A. 8 


72 


X 


A5 


1.7 


73 


X 


AA 


3. A 


7^ 


X 


SI 


3.0 


75 


X 


S2 


2.0 


76 


X 


S3 


3.5 


77 


X 


SA 


l.A 


78 






/ 


79 


XXXXXXXXXXXXXXXXXXXMXXXXXXXXXXXXXXXX)#XXXXX)<XXXXXXXXXXXXXXXXXXXXXXXXXXX 


80 




NEPROB(I) probability of no crew error task i - detm by INE 


81 








82 




/ 




85 


X 


G1 


.992200 


8^4 


X 


G2 


.9906A0 


85 


X 


G5 


.995320 


86 


X 


GA 


.9958A0 


87 


X 


AG 


.97A000 


88 




A1 


.9888A6 


89 


X 


A2 


.99087A 


90 


X 


A3 


.987525 


91 


X 


AA 


.982255 


92 


X 


SI 


.995652 


95 


X 


S2 


.99599A 


9^ 


X 


S3 


.99126A 


95 


X 


SA 


.991810 


96 






/ 


97 








98 


xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 


99 




EPROB(I) probability of crew error on task i ~ detm by INE 


100 








101 




/ 




102 


X 


G1 


.007800 


105 


X 


G2 


.009560 


10^ 


X 


G5 


.00A680 


105 


X 


GA 


.00A160 


106 


X 


AG 


.026000 


107 




A1 


.01115A 


108 


X 


A2 


.009126 


109 


X 


A3 


.012675 


no 


X 


AA 


.0177A5 


111 


X 


SI 


.00A368 


112 


X 


S2 


.006006 


115 


X 


S3 


.008736 


11^ 


X 


SA - 


.008190 
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The Alternative Model (continued) 



GPNLP2 



115 

116 
117 






118 


X 


these costs ere derived using 2 criteria. First - mean cost is based 


119 


X 


on data 


from firing tables assuming range to target 6000 meters^ and 


120 


X 


beta distribution (pert estimation techniques). Second - max cost 


121 


X 


is based 


1 on data derived from firing tables. 


122 








125 




MCOST(I) 


mean cost of error i - detm by PERT Beta distn techn 


12^ 








125 








126 


X 


SET 2 (max costs) 


127 




/ 




128 


X 


G1 


2^00 


129 


X 


G2 


200 


150 


X 


G5 


2^00 


151 


X 


G^ 


^00 


152 


X 


AG 


2600. 


155 




A1 


2^00 . 


15^ 


X 


A2 


100 . 


155 


X 


A5 


120. 


156 


X 


AA 


2500 . 


157 


X 


SI 


2^00 . 


138 


X 


S2 


2000. 


159 


X 


S5 


2500. 


1^0 


X 


SA 


150. 


1^1 






/ 


H2 








1^5 


xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 






SCALARS 




1^5 




RESPTIME total time allowed for guns to shoot / 50.0/ 


1^6 




BPARK 


number meters in which aimpoint must be / 000/ 


1^7 




TIMEWT 


weight placed on time pos devn in OBJ / 1.0/ 


H8 




BPHT 


weight placed on error pos devn in OBJ / 600 /; 


M9 








150 


xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 


151 




VARIABLES 




152 




XCI) 


time to be spent inspecting each task in battery 


155 




TOFF 


deviation from the time goal constraint 


15^ 




BPOFF 


deviation from the ballpark goal constraint 


155 




DEVIATION deviation from the objective function; 


156 








157 


xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 


158 




POSITIVE 


VARIABLES 


159 




X(I), 


TPOS, TNEG, BPPOS, BPNEG ; 


160 








161 


xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 


162 




EQUATIONS 




165 




OBJDEF 


achievement function 


16^ 




TLIMIT 


time limit 


165 




DLIMIT 


distance limit; 



166 

167 

168 

169 

170 



XXKXXKXKXXXXXXKXXXXXXXXXXXXKKXXXXKKKXXXXKXXXXXXXXXXXXXXXXXXXXXXXXXX 

X MINIMIZE 

OBJDEF. . DEVIATION =E= 
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The Alternative Model (continued) 



GPNLP2 



171 

172 

173 
17^1 

175 

176 

177 

178 

179 

180 
181 
182 
183 
18 ^ 

185 

186 

187 

188 

189 

190 

191 

192 

193 



( TIMEWT X TPOS ) + ( BPWT X BPPOS 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
X SUBJECT TO 

TLIMIT,. SUM ( I, X(I) ) - TPOS + TNEG =E= RESPTIME ; 

DLIMIT. . SUM ( I, 

( 1 - SQRT( (SQR(EPROBd))) x (X(I)-MIN(I)) / (MAX( I )-MIN( I ) ) ) 

- NEPROB(I) ) X MCOST(I) ) -BPPOS + BPNEG =E= BPARK ; 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
X ADDITIONAL CONSTRAINTS involving min end max times allowed. These 
X keep X in the feasible region as problem iterates# otherwise stops 

X.LO(I) = MIN(I)X1 .01; 

X.UP(I) = MAX(I); 

X if region is convex# then no matter where starts# soln should be same 
X Need to start internal because funny things happen on edges 

X.L(I) = MINCDxl .02 

; 

MODEL GPNLP2 /ALL/; 

SOLVE GPNLP2 USING NLP MINIMIZING DEVIATION; 
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3. The Output 

a* The Current Model 



GPNLP2 

SOLUTION REPORT SOLVE GPNLP2 USING NLP FROM LINE 150 



SOLVE SUMMARY 





MODEL 


GPNLP2 




OBJECTIVE 


DEVIATION 




TYPE 


NLP 




DIRECTION 


MINIMIZE 




SOLVER 


MINOS5 




FROM LINE 


150 




SOLVER 


STATUS 


1 


NORMAL COMPLETION 






MODEL 


STATUS 


2 


LOCALLY OPTIMAL 






OBJECTIVE VALUE 




0.0151 





RESOURCE USAGE, LIMIT 


0 . 


,263 


1000.000 


ITERATION COUNT, LIMIT 


23 




1000 


EVALUATION ERRORS 


0 




0 


MINOS VERSION 


5.0 


APR 


198A 



COURTESY OF B. A. MURTAGH AND M. A. SAUNDERS, 
DEPARTMENT OF OPERATIONS RESEARCH, 
STANFORD UNIVERSITY, 

STANFORD CALIFORNIA 9A505 U.S.A. 



MORK SPACE NEEDED (ESTIMATE) 
WORK SPACE AVAILABLE 
(MAXIMUM OBTAINABLE 



825 MORDS. 
1179 MORDS. 
16^97^ WORDS. ) 



EXIT — OPTIMAL SOLUTION FOUND 



MAJOR ITERATIONS 
NORM RG / NORM PI 
TOTAL USED 
MIN0S5 TIME 



1 

5.001E-09 

0.28 UNITS 

0.17 (INTERPRETER 



0.05) 



LOWER LEVEL UPPER 



— 


EQU 


OBJDEF 




0.1300 


0.1300 


0.1300 


“ 


EQU 


TLIMIT 




-INF 


30.0000 


30.0000 


OBJDEF 


ACHIEVEMENT 


FUNCTION 






TLIMIT 


TIME 


LIMIT 










VAR 


X 


TIME 


TO BE SPENT 


INSPECTING EACH 


TASK IN BATTERY 






LOWER 




LEVEL 


UPPER 


MARGINAL 


G1 




0.7070 




2.2A27 


2.8000 


EPS 


G2 




0.7070 




2.5660 


3.2000 


EPS 


G5 




0.7070 




1.0000 


1.0000 


-0.0056 


G^ 




0.7070 




1.6000 


1.6000 


-0.0001 


AG 




2.0200 




7.2167 


8.9000 


, 


A1 




0.7070 




1 .8000 


1.8000 


-0.0029 


A2 




0.7070 




1.7816 


A .8000 


EPS 


A5 




0.7070 




1 .7000 


1.7000 


-0.00^2 


A^ 




0.7070 




3.A000 


3.A000 


-0.0011 


SI 




0.7070 




1 .1A17 


3.0000 


EPS 


S2 




0.7070 




2.0000 


2.0000 


-0.0001 


S5 




0.7070 




2.1513 


•3.5000 


EPS 


SA 




0.7070 




1 . AOOO 


1 . AOOO 


-0.0037 










LOWER 


LEVEL 


UPPER 


— 


VAR 


DEVIATION 




-INF 


0.0151 


+ INF 



MARGINAL 

-1 . 0000 
- 0.0022 



MARGINAL 
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b. The Alternative Model First Tier 



GPNLP2 

SOLUTION REPORT SOLVE GPNLP2 USING NLP FROM LINE 193 



SOLVE SUMMARY 



MODEL GPNLP2 
TYPE NLP 
SOLVER MIN0S5 



OBJECTIVE DEVIATION 
DIRECTION MINIMIZE 
FROM LINE 193 



SOLVER STATUS 
MODEL STATUS 
XXKK OBJECTIVE VALUE 



1 NORMAL COMPLETION 

2 LOCALLY OPTIMAL 

0.0 



RESOURCE USAGE, LIMIT 
ITERATION COUNT, LIMIT 
EVALUATIOfI ERRORS 



0.<^19 1000.000 

<\ 1000 

0 0 



MINOS VERSION 5.0 APR 198A 



COURTESY OF B. A. MURTAGH AND M. A. SAUNDERS, 
DEPARTMENT OF OPERATIONS RESEARCH, 
STANFORD UNIVERSITY, 

STANFORD CALIFORNIA 9^305 U.S.A. 



HORK SPACE NEEDED (ESTIMATE) 
WORK SPACE AVAILABLE 
(MAXIMUM OBTAINABLE 



350 HORDS. 
HORDS. 

16^97^ HORDS. ) 



EXIT 



OPTIMAL SOLUTION FOUND 



MAJOR ITERATIONS 
NORM RG / NORM PI 
TOTAL USED 
MINOS5 TIME 



^6 

O.OOOE+00 

0.^3 UNITS 

0.39 (INTERPRETER 



0 . 01 ) 



LONER LEVEL UPPER MARGINAL 



-- 


-- 


EQU 


OBJDEF 


, 






-1 .0000 


-- 


— 


EQU 


TLIMIT 


50.0000 


30! 0000 


50.0000 


EPS 


— 


— 


EQU 


DLIMIT 


-26.7696 


-26.7696 


-26.7696 


EPS 




OBJDEF 


ACHIEVEMENT FUNCTION 










TLIMIT 


TIME 


LIMIT 










DLIMIT 


DISTANCE LIMIT 








— 


— 


VAR 


X 


TIME TO BE SPENT 


INSPECTING EACH 


TASK IN BATTERY 










LOWER 


LEVEL 


UPPER 


MARGINAL 




A1 






0.7070 


1.8000 


1.8000 














LONER 


LEVEL 


UPPER 


MARGINAL 


— 


— 


VAR 


DEVIATION 


-INF 




+ INF 




-- 


— 


VAR 


TPOS 






+ INF 


1 !oooo 


-- 




VAR 


TNEG 




28!2000 


+ INF 






-- 


VAR 


BPPOS 






+ INF 


600.0000 


— 


— 


VAR 


BPNEG 






+ 1NF 


EPS 
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c 



The Alternative Model’s Second Tier 



GPMLP2 

SOLUTION REPORT SOLVE GPNLP2 USING NLP FROM LINE 191 





S 0 L V 


E 


S U M M A R 


Y 


MODEL 


GPNLP2 




OBJECTIVE 


DEVIATION 


TYPE 


NLP 




DIRECTION 


MINIMIZE 


SOLVER 


MIN0S5 




FROM LINE 


191 


xxxx SOLVER 


STATUS 


1 


NORMAL COMPLETION 




xxxx MODEL 


STATUS 


2 


LOCALLY OPTIMAL 




xxKit OBJECTIVE VALUE 




0.0 




RESOURCE USAGE, LIMIT 




0.582 


1000.000 


ITERATION 


COUNT, LIMIT 




25 


1000 



EVALUATION ERRORS 0 0 

MINOS VERSION 5.0 APR 198A 



COURTESY OF B. A. MURTAGH AND M. A. SAUNDERS, 
DEPARTMENT OF OPERATIONS RESEARCH, 
STANFORD UNIVERSITY, 

STANFORD CALIFORNIA 9A305 U.S.A. 



HORK SPACE NEEDED (ESTIMATE) 
HORK SPACE AVAILABLE 
(MAXIMUM OBTAINABLE 



60A HORDS. 
858 HORDS. 
16A97A HORDS.) 



EXIT “ OPTIMAL SOLUTION FOUND 



MAJOR ITERATIONS 
NORM RG / NORM PI 
TOTAL USED 
MIfiOSS TIME 



O.OOOE+00 

0.60 UNITS 

0.51 (INTERPRETER 



0.06) 







LOWER 


LEVEL 


UPPER 


MARGINAL 


EQU 


OBJDEF 








-1 .0000 


EQU 


TLIMIT 


2s!2000 


28^2000 


28!2000 


EPS 


EQU 


DLIMIT 


-186 .2A97 


-186 .2A97 


-186 .2A97 


EPS 


OBJDEF 


ACHIEVEMENT 


FUNCTION 








TLIMIT 


TIME LIMIT 










DLIMIT 


DISTANCE LIMIT 








VAR 


X TIME 


TO BE SPENT 


INSPECTING EACH 


TASK IN BATTERY 






LOWER 


LEVEL 


UPPER 


MARGINAL 




G1 


0.7070 


2.8000 


2.8000 


EPS 




G5 


0.7070 


1 .0000 


1 .0000 


EPS 




AG 


2.0200 


8.9000 


8.9000 


, 




AA 


0.7070 


3.A000 


3.A000 


EPS 




SI 


0.7070 


3.0000 


3.0000 


EPS 




S2 


0.7070 


2.0000 


2.0000 


EPS 




S3 


0.7070 


3.5000 


3.5000 


EPS 








LOWER 


. LEVEL 


UPPER 


MARGINAL 


VAR 


DEVIATION 


-INF 




+ INF 




VAR 


TPOS 






+ INF 


1 !oooo 


VAR 


TNEG 




3!6000 


+ INF 




VAR 


BPPOS 






+ II^F 


600 .0000 


VAR 


BPNEG 






+ INF 


EPS 
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d. The Alternative Model’s Third Tier 



GPNLP2 

SOLUTION REPORT SOLVE GPNLP2 USING NLP FROM LINE 208 



SOLVE SUMMARY 





MODEL 


GPNLP2 




OBJECTIVE 


DEVIATION 




TYPE 


NLP 




DIRECTION 


MINIMIZE 




SOLVER 


MINOS5 




FROM LINE 


208 




SOLVER 


STATUS 


1 


NORMAL COMPLETION 






MODEL 


STATUS 


2 


LOCALLY OPTIMAL 






OBJECTIVE VALUE 




A.810A 





RESOURCE USAGE, LIMIT 0.309 1000.000 

ITERATION COUNT, LIMIT 29 1000 

EVALUATION ERRORS 0 0 



MINOS VERSION 5.0 APR 198A 



COURTESY OF B. A. MURTAGH AND M. A. SAUNDERS, 
DEPARTMENT OF OPERATIONS RESEARCH, 
STANFORD UNIVERSITY, 

STANFORD CALIFORNIA 9A305 U.S.A. 



MORK SPACE NEEDED (ESTIMATE) 
NORK SPACE AVAILABLE 
(MAXIMUM OBTAINABLE 



511 WORDS. 
709 WORDS. 
16A97A WORDS.) 



EXIT -- OPTIMAL SOLUTION FOUND 



MAJOR ITERATIONS 


13 








NORM RG 


/ NORM PI 


8.A63E-08 








TOTAL USED 


0.32 UNITS 








MIN0S5 


TIME 


0.25 (INTERPRETER - 0 


.03) 








LOWER 


LEVEL 


UPPER 


MARGINAL 


EQU 


OBJDEF 








-1.0000 


EQU 


TLIMIT 


3!6000 


3 I 6 OOO 


3.6000 


-1 . 0000 


EQU 


DLIMIT 


-7.1981 


-7.1981 


-7 . 1981 


-1.0000 


OBJDEF 


ACHIEVEMENT FUNCTION 








TLIMIT 


TIME 


LIMIT 








DLIMIT 


DISTANCE LIMIT 








VAR 


x 


TIME TO BE SPENT 


INSPECTING EACH 


TASK IN BATTERY 






LOWER 


LEVEL 


UPPER 


MARGINAL 




G2 


0.7070 


1 . 050A 


3.2000 


EPS 




GA 


0.7070 


1 . A691 


1.6000 






A2 


0.7070 


0.7508 


A. 8000 


EPS 




A3 


0.7070 


1.278^ 


1.7000 


EPS 




SA 


0.7070 


1.2390 


1 .AOOO 


EPS 









LOWER 


LEVEL 


UPPER 


MARGINAL 


VAR 


DEVIATION 


-INF 


^.810^ 


+ INF 


. 


VAR 


TPOS 




2.1877 


+ IfJF 


EPS 


VAR 


TNEG 






+ IIIF 


1.0000 


VAR 


BPPOS 




2.6227 


+ INF 


. 


VAR 


BPNEG 


. 


, 


4INF 


1 .0000 
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APPENDIX F 



GLOSSARY 



A. Human Reliability Terminology 

1. "Consequence of interest (COI)" is analogous to 
measure of effectiveness (MOE). MOE is used when 
the concern is maximizing effectiveness of some 
positive performance whereas COI is used when the 
concern is minimizing consequences of some 
negative performance (for example: human error). 

2. "Human error" is any member of a set of human 
actions that exceeds some limit of acceptability. 
The set of actions must be defined and the 
tolerance limits must be established for there to 
be human error. [Ref 10, p.6] 

3. "Human error probability" is the chance that a 
given task will not be successfully completed by 
personnel within a required minimum time (if a 
time requirement exists). This is the reciprocal 
of human reliability. 

4. "Human reliability" is the probability that a task 
will be completed successfully by personnel within 
a specified minimum time (if a time requirement 
exists). [Ref 1, p. 221] 

5. "Worker" is the human on the production line who 
performs the error-prone tasks that potentially 
effect the finished product. A mistake by a 
worker that potentially leads to a flawed product 
is referred to as an "error". 

6. "Inspector" is the human tasked to find faulty 
finished products before they leave the production 
line. A mistake by an inspector that leads to a 
flawed product is referred to as a "failure". 

7. "System error" is a worker error which evades 
detection by the inspector and has a cost that is 
of consequence of interest to the system. A 
system error is also referred to as a "flawed 
product" . 
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B. Artillery Terminology 



1. A "chief" is the senior man in a crew of men who 
operate a gun. 

2. "Fire(s)", when used as a noun, refers to the 
shooting of artillery munitions. 

3. A "fire mission" is an order from higher authority 
to shoot one or more rounds of artillery at a 
target. 

4. A "gun" refers to a single artillery cannon weapon 
system. 

5. "Gunnery" refers to the group of skills required 
of crewmen to fire their weapon. 

6. "Maneuver", when used as a noun, refers to 
infantry or armor elements to which the artillery 
provides fire support. 

7. A "round" is synonymous with the term projectile. 

8. A "unit" refers to a battery of artillery. A 
battery is either six or eight cannon weapon 
systems . 
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APPENDIX G 



SURVEYS ADMINISTERED TO RESPONDENTS 
The collection of the expert judgments from operational 
personnel was key to solving the illustrative problem. The 
error probability survey and the time use survey were the two 
surveys given to the respondents. Respondents were briefed 
in person prior to working on the survey. The surveys took 
approximately 45 minutes to complete. 

The error probability survey requires a gun chief to 
compare pairs of errors as to their relative likelihood of 
occurrence. The comparisons are made among similar type 
errors within the hierarchical areas defined in Figure 9 in 
Appendix A. The gun chief is also required to make one direct 
numerical estimate on the likelihood that his section sets any 
type of error. The process by which these survey results are 
turned into probabilities is discussed in Appendix A. The 
method by which the respondent's judgments are checked for 
consistency is discussed in Appendix D. 

The time use survey requires a gun chief to compare the 
amount of time taken to perform one inspection task relative 
another. Comparisons are made among similar type tasks within 
hierarchical areas defined in Figure 9 in Appendix A. One 
inspection task which is routinely done in a relatively 
uniform manner among all gun chiefs is the inspection of the 
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gunner's sight picture on the gun's aiming reference point. 
It is assumed that this task takes 2.0 seconds to perform. 
Using that known value, all other visual inspection tasks are 
scaled accordingly. The process by which the tasks are scaled 
from the survey results is discussed in Appendix C. 
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1. The Error Probability Survey. 

FREQUENCY BY WHICH ERRORS OCCUR 
(gunner, asst gunner, ammo handlers, FDC) 

Which crew element (the one on the left or the one on the 
right) is most likely to make an error that you must have 
corrected prior to firing? How many times more likely is the 
more error-prone element to make an error than the less error- 
prone element? (Assume daylight firing, shell HE, fuze TIME, 
Low Angle; with collimator, gunner's quadrant, and safety T. ) 
( make one "X" per comparison below ) 

WHICH IS MORE LIKELY ? 



how many times 

ERROR as likely? ERROR 

in: >9 8 7 6 5 4 3 2 2 3 4 5 6 7 8 >9 in: 



gunner 
































asst 

gunner 


gunner 
































ammo 

handlers 


gunner 
































FDC / 
safety T 


asst 

gunner 
































ammo 

handlers 


asst 

gunner 
































FDC / 
safety T 


ammo 

handlers 
















= - 
















FDC / 
safety T 



Indicate below the number of "good" missions to each 
mission that you must get involved in making a correction on 
either the gunner, asst gunner, ammo handlers, or FDC (a 
"faulty" mission is one in which you must get involved to keep 
shooting accurate. ) 





How many times as likely 
are good missions to each faulty mission? 




1 


1 2 3 4 5 

till 


6 7 8 9 

1 1 1 


10 20 30 
1 1 


40 50 
1 


60 


>70 


faulty 


11**1 ** 1 ** 
1 1 1 1 


** 1 ** 1 ** 1 ** 
1 1 I 


** 1 ** 1 ** 
1 1 


** 1 ** 
1 




good 


mission 


1 1 1 1 


1 1 1 


1 1 


1 




mission 
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GUNNER ERRORS 



Which error (the one on the left or the one on the right) is 

most likely to occur (prior to your safety inspection) in your gun 

section? How many more times likely is the more frequent event 

than the less frequent event? (Assume daylight firing, good 

conditions, shell HE, fuze TIME, Low Angle; with collimator, 

gunner's quadrant, and safety T. ) 

( make one "X" per comparison below ) 

( Mark only six "X"s on this page ) 

WHICH IS 
MORE LIKELY ? 



how many times 

ERROR as likely? ERROR 

in: >9 8 7 6 5 4 3 2 2 3 4 5 6 7 8 >9 in: 



sight 

picture 
















= = 
















bubbles 


sight 

picture 
















= = 
















counter 


sight 

picture 
















= = 
















piece 

corrn 


bubbles 
















= = 
















”df 

counter 


bubbles 
















= = 
















piece 

corrn 


df 

counter 
















= = 
















piece 

corrn 
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ASSISTANT GUNNER ERRORS 



Which error (the one on the left or the one on the right) is 

most likely to occur (prior to your safety inspection) in your gun 

section? How many more times likely is the more frequent event 

than the less frequent event? (Assume daylight firing, good 

conditions, shell HE, fuze TIME, Low Angle; with collimator and 

gunner's quadrant.) 

( make one "X" per comparison below ) 

( Mark only six "X"s on this page ) 

WHICH IS 
MORE LIKELY ? 



how many times 

ERROR as likely? ERROR 

in: >9 8 7 6 5 4 3 2 2 3 4 5 6 7 8 >9 in: 



counter 

counter 

counter 

elev 

bubble 

elev 

bubble 

cr Ivl 
bubble 



elev 

bubble 

cr Ivl 
bubble 

piece 

corrn 

cr Ivl 
bubble 

piece 

corrn 

piece 

corrn 
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AMMUNITION HANDLERS ERRORS 



Which error (the one on the left or the one on the right) is 

most likely to occur (prior to your safety inspection) in your gun 

section? How many more times likely is the more frequent event 

than the less frequent event? (Assume daylight firing, good 

conditions, shell HE, fuze TIME, Low Angle; with collimator, 

gunner's quadrant, and safety T. ) 

( make one "X" per comparison below ) 

( Mark only six "X"s on this page ) 

WHICH IS 
MORE LIKELY ? 



how many times 

ERROR as likely? ERROR 

in: >9 8 7 6 5 4 3 2 2 3 4 5 6 7 8 >9 in: 
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FDC DATA ERRORS 



(data is off of Safety T) 



Which error (the one on the left or the one on the right) 
is most likely to occur? How many more times likely is the more 
frequent event than the less frequent event? (Assume daylight 
firing, good conditions, shell HE, fuze TIME, Low Angle; with 
collimator, gunner's quadrant, and safety T. ) 

( make one "X" per comparison below ) 

( Mark only six "X"s on this page ) 

WHICH IS 
MORE LIKELY ? 



how many times 

ERROR as likely? ERROR 

in: >9 8 7 6 5 4 3 2 2 3 4 5 6 7 8 >9 in: 
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2. The Time Use Survey. 

COMPARISON OF TIME TAKEN TO PERFORM 
SAFETY CHECKS ON CREW 

Which check (the one on the left or the one on the right) 
consumes more of your time? How many more seconds does it take to 
make the longer check verses the shorter check in each of the six 
comparisons listed below? (Assume daylight firing, good 
conditions, shell HE, fuze TIME, Low Angle; with collimator, 
gunner's quadrant, and safety T. ) 

( make one "X" per comparison below ) 

WHICH CHECK IS MORE 
TIME CONSUMING 



increased number of seconds it takes to do one task vs the other 
#sec >8 7654321 1234567 >8 #sec 
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DEFLECTION CHECKS 



Which check (the one on the left or the one on the right) 

consumes more of your time? How many more seconds does it take to 

make the longer check verses the shorter check in each of the six 

comparisons listed below? (Assume daylight firing, good 

conditions, shell HE, fuze TIME, Low Angle; with collimator, 

gunner's quadrant, and safety T. ) 

( make one "X" per comparison below ) 

( only mark six "X"s on this form ) 

WHICH CHECK IS MORE 
TIME CONSUMING 

increased number of seconds it takes to do one task vs the other 
#sec >8 7654321 1234567 >8 #sec 



sight 

picture 
















= = 
















bubbles 


sight 

picture 
















= = 
















“df 

counter 


sight 

picture 
















= = 
















piece 

corrn 


bubbles 
















= = 
















“df 

counter 


bubbles 
















= = 
















piece 

corrn 


df 

counter 
















= = 
















piece 

corrn 

















1 
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AMMUNITION CHECKS 



Which check (the one on the left or the one on the right) 
consumes more of your time? How many more seconds does it take to 
make the longer check verses the shorter check in each of the six 
comparisons listed below? (Assume daylight firing, good 

conditions, shell HE, fuze TIME, Low Angle; with collimator, 
gunner's quadrant, and safety T. ) 

( make one "X" per comparison below ) 

WHICH CHECK IS MORE 
TIME CONSUMING 



increased number of seconds it takes to do one task vs the other 
#sec >8 7654321 1234567 >8 #sec 



charge 
















= = 
















time 

setting 


charge 
















= = 
















fuze 

type 


charge 
















= = 
















shell 

type/wt/lot 


time 

setting 
















= = 
















fuze 

type 


time 

setting 
















= = 
















shell 

type/wt/lot 


fuze 

type 
















zz zz 
















shell 

type/wt/lot 
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FDC DATA CHECKS 
(Safety T violations) 

Which check (the one on the left or the one on the right) 
consumes more of your time? How many more seconds does it take to 
make the longer check verses the shorter check in each of the six 
comparisons listed below? (Assume daylight firing, good 

conditions, shell HE, fuze TIME, Low Angle; with collimator, 
gunner's quadrant, and safety T. ) 

( make one "X" per comparison below ) 

WHICH CHECK IS MORE 
TIME CONSUMING 



increased number of seconds it takes to do one task vs the other 
#sec >8 7654321 1234567 >8 #sec 



deflec- 

tion 
















= = 
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deflec- 
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time for 
fuze 
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